Share

Objective Measurement of Subjective Phenomena

6. Problems in Measuring Constructs

Biases in Measurement

Biases represent systematic influences on item or scale scores that are unrelated to the construct to be measured (Paulhus, 1991). Bias falls under the rubric of construct-irrelevant variance.

Bias in Self-Report

Self-reports are associated with several forms of potential bias:

Acquiescence: “yea-saying” or “nay-saying”.

  • “Yea-saying” – tendency to endorse items regardless of item content.
  • “Nay-saying” – tendency to refuse to endorse items regardless of item content.
Solution: Balance item content on a scale, with about half of the items positively worded (so a high score on an item indicates high standing on the trait) and about half of the items negatively worded (so a high score on an item indicates a low standing on the trait).

Figure 2

Figure of acquiescence as discussed in text.

Social desirability: tendency to respond positively to positively valued item content and, conversely, to respond negatively to negatively valued item content (Paulhus, 1984).

Solution: During test development, administer a social desirability scale along with items for scales being developed. Then, during item analyses, one can discard items that are too saturated with social desirability (i.e., that correlate too highly with social desirability).

Figure 3

Figure of acquiescence as discussed in text.

Extremity (Peabody, 1962):

  • Consider a 1-to-9 scale ranging from “strongly disagree” to “strongly agree”.
  • Some persons tend to use only the most extreme values (tend to answer questions using item options 1 or 2 when disagree, or 8 or 9 when agree), so do not use the middle values 3-7 very much.
  • Other individuals tend to refrain from using the extreme values even when they very strongly agree or disagree.
Solution: Make sure to have balanced set of items with regard to direction of wording (positively worded and negatively worded), as this will tend to decrease extremity bias effects as well as acquiescence bias effects.

Figure 4

Figure of acquiescence as discussed in text.

Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17-59). New York: Academic.
Paulhus, D. L. (1984). Two-component models of social desirable responding. Journal of Personality and Social Psychology, 46, 598-609.