Objective Measurement of Subjective Phenomena

6. Problems in Measuring Constructs

Biases in Measurement

Bias in other-person report

Biases may also arise when other persons (e.g., parents, teachers, observers) provide reports on an individual (Campbell & Fiske, 1959; Eid & Diener, 2006).

  • Anchoring: Bias in ratings due to prior information.

    If raters are informed that “many people exhibit a given type of behavior” (or tend not to exhibit that behavior), this may influence their judgments of other persons’ actions and behaviors.

    Solution: To avoid idiosyncratic anchors, provide explicit anchors for raters.

Example 24

Anchoring example:

An example might be: “In comparison to other persons of the same age and sex, please rate the person on the following scales: …”.

  • Halo effects: Bias that yields a generalized positive or negative evaluation.

    Halo effects arise in many situations in which the rater knows the person being rated rather well.

    Solution: Obtain ratings of each target from multiple raters. Thus, one could obtain ratings by multiple supervisors of each target subordinate, or could obtain ratings by both mother and father of target children. Sophisticated methods of analysis can then be used to model the effects of halo bias, but only if two or more raters have been used to obtain ratings.

Example 25

Halo effect examples:

Supervisors may have generalized positive or negative biases regarding their subordinates. Or, parents may have generalized positive or negative biases about one or another of their children.
Eid, M., & Diener, E. (Eds.). (2006). Handbook of multimethod measurement in psychology. Washington, DC: American Psychological Association.