Objective Measurement of Subjective Phenomena

Correlation between two separate forms or scales developed to assess the same construct.

This coefficient tells us: precision of measurement of each form – how accurately individuals are characterized at a single point in time by each form.

Parallel forms are difficult to develop, must meet exacting standards. They must have:

• Equal means
• Equal standard deviations
• Equal reliabilities
• Equal correlations with outside variables

The correlation between the two parallel forms is the parallel forms reliability for either of the parallel forms, using the other as parallel form. For example, assume the presence of Forms A and B to measure a given construct. If a sample of individuals is administered both forms and scores on the two forms correlate .84, then the parallel forms reliability for Form A is .84 (using Form B as the basis of this claim) and the parallel forms reliability for Form B is .84 (using Form A as the basis of this claim).

Correlation between two scores constructed as (random) halves of a set of items on a scale (e.g., the correlation of the sum score on even-numbered items with the sum score on odd-numbered items).

This coefficient tells us: precision of measurement of the form in question at the given time of measurement.

Assume the presence of two split halves – Half E (for even-numbered items) and Half O (for odd-numbered items). Assume that the correlation between Half E and Half O is signified as rHO, and rHO = .80. This reliability of .80 is the reliability of each half of the scale. But, because the scale is twice as long as either half (if each half of the scale consists of 6 items, the total scale has 12 items), this reliability must be adjusted upward to estimate the reliability of the total scale, or rxx. To do so, one must use a special form of the Spearman-Brown prophecy formula.

In the preceding formula, the term rxx(2) has a (2) in its subscript to indicate that it is an estimated reliability assuming that the total scale is 2 times as long as either form that was correlated.

Touted as average of all possible split-halves, based on the set of items responses, specifically the variance-covariance matrix among items

This coefficient tells us: precision of measurement of the form in question at the given time of measurement.

• Kuder-Richardson 20, 21 for dichotomous items
• Coefficient alpha
• Coefficient omega

Correlation between scores for the same individuals at two different points in time.

This coefficient tells us: whether individual differences are similar at the two points in time.

• Trait construct – ideally, would have high stability
• State construct – ideally, would have low(er) stability

Interval is important.

• Interval too short – too much “memory” effect – person remembers how s/he answered last time, answers the same the next time
• Interval too long – construct (true score) has changed, and change in construct would be treated as error variance

Correlation between scores for same individuals at two points in time, in which individuals take one parallel form at one time and the different parallel form at the second time.

This coefficient tells us: whether individual differences are similar at the two points in time even when precise content of measure changes.

Have 2 (or more) raters provide ratings of each person in a sample (e.g., ratings of aggressiveness of each child based on observing the children interacting on a playground for several days). Then correlate ratings by rater 1 with ratings by rater 2.

This coefficient tells us: whether two (or more) individuals have similar views of differences among persons being rated (i.e., whether they “see” the same things in people).

Problem: interrater reliability may be high, even though certain other forms of reliability are low.