Congeneric reliability explained

In statistical models applied to psychometrics, congeneric reliability

\rho _{C} ("rho C")^[1] a single-administration test score reliability (i.e., the reliability of persons over items holding occasion fixed) coefficient, commonly referred to as composite reliability, construct reliability, and coefficient omega. \rho _{C} is a structural equation model (SEM)-based reliability coefficients and is obtained from on a unidimensional model. \rho _{C} is the second most commonly used reliability factor after tau-equivalent reliability(\rho _{T} ; also known as Cronbach's alpha), and is often recommended as its alternative.

History and names

A quantity similar (but not mathematically equivalent) to congeneric reliability first appears in the appendix to McDonald's 1970 paper on factor analysis, labeled

\theta .^[2] In McDonald's work, the new quantity is primarily a mathematical convenience: a well-behaved intermediate that separates two values. Seemingly unaware of McDonald's work, Jöreskog first analyzed a quantity equivalent to congeneric reliability in a paper the following year.^[3] Jöreskog defined congeneric reliability (now labeled ρ) with coordinate-free notation, and three years later, Werts gave the modern, coordinatized formula for the same.^[4] Both of the latter two papers named the new quantity simply "reliability". The modern name originates with Jöreskog's name for the model whence he derived \rho _{C} : a "congeneric model".^{[5] [6]}

Applied statisticians have subsequently coined many names for

{\rho} _{C} . "Composite reliability" emphasizes that {\rho} _{C} measures the statistical reliability of composite scores.^[7] As psychology calls "constructs" any latent characteristics only measurable through composite scores,^[8] {\rho} _{C} has also been called "construct reliability".^[9] Following McDonald's more recent expository work on testing theory, some SEM-based reliability coefficients, including congeneric reliability, are referred to as "reliability coefficient \omega ", often without a definition.^{[10] [11]}

Formula and calculation

Congeneric reliability applies to datasets of vectors: each row in the dataset is a list of numerical scores corresponding to one individual. The congeneric model supposes that there is a single underlying property ("factor") of the individual, such that each numerical score is a noisy measurement of . Moreover, that the relationship between and is approximately linear: there exist (non-random) vectors and such that $$X\_i=\backslash lambda\_iF+\backslash mu\_i+E\_i\backslash text$$ where is a statistically independent noise term.

In this context, is often referred to as the factor loading on item .

Because and are free parameters, the model exhibits affine invariance, and may be normalized to mean and variance without loss of generality. The fraction of variance explained in item by is then simply $$\backslash rho\_i=\backslash frac\backslash text$$ More generally, given any covector, the proportion of variance in explained by is $$\backslash rho=\backslash frac\backslash text$$ which is maximized when .

is this proportion of explained variance in the case where (all components of equally important): $$\backslash rho\_C\; =\; \backslash frac$$

Example

Fitted/implied covariance matrix

	X _{1}	X _{2}	X _{3}	X _{4}
X _{1}	10 . 00
X _{2}	4 . 42	11 . 00
X _{3}	4 . 98	5 . 71	12 . 00
X _{4}	6 . 98	7 . 99	9 . 01	13 . 00
\Sigma	124 . 23 = \Sigma _{diagonal} + 2 x \Sigma _{subdiagonal}

These are the estimates of the factor loadings and errors:

Factor loadings and errors

	\hat{ λ } _{i}	\hat{\sigma} ^{2} _{e _{i}}
X _{1}	1 . 96	6 . 13
X _{2}	2 . 25	5 . 92
X _{3}	2 . 53	5 . 56
X _{4}	3 . 55	. 37
\Sigma	10 . 30	18 . 01
\Sigma ^{2}	106 . 22

\hat{\rho} _{C} = (\left( \sum ^{k} _{i =1} \hat{ λ)/(_{i}) \right) ^{2} }{ \hat{\sigma} ^{2} _{X} } = (106 . 22)/(124 . 23) = . 8550 \hat{\rho} _{C} = (\left( \sum ^{k} _{i =1} \hat{ λ)/(_{i}) \right) ^{2} }{ \left( \sum ^{k} _{i =1} \hat{ λ } _{i} \right) ^{2} + \sum ^{k} _{i =1} \hat{\sigma} ^{2} _{e _{i}} } = (106 . 22)/(106 . 22+18 . 01) = . 8550

Compare this value with the value of applying tau-equivalent reliability to the same data.

Related coefficients

Tau-equivalent reliability (

\rho _{T} ), which has traditionally been called "Cronbach's \alpha ", assumes that all factor loadings are equal (i.e. λ _{1=λ} _{2=...=λ} _{k} ). In reality, this is rarely the case and, thus, it systematically underestimates the reliability. In contrast, congeneric reliability (\rho _{C} ) explicitly acknowledges the existence of different factor loadings. According to Bagozzi & Yi (1988), \rho _{C} should have a value of at least around 0.6.^[12] Often, higher values are desirable. However, such values should not be misunderstood as strict cutoff boundaries between "good" and "bad".^[13] Moreover, \rho _{C} values close to 1 might indicate that items are too similar. Another property of a "good" measurement model besides reliability is construct validity.

A related coefficient is average variance extracted.

References

1. Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239
2. Although McDonald, R. P. (1985). Factor analysis and related methods. Hillsdale, NJ: Lawrence Erlbaum and (1999). Test theory. Mahwah, NJ: Lawrence Erlbaum claim that invented congeneric reliability, there is a subtle difference between the formula given there and the modern one. As discussed in, McDonald's denominator totals observed covariances, but the modern definition divides by the sum of fitted covariances.
3. Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109–133. https://doi.org/10.1007/BF02291393
4. Werts, C. E., Linn, R. L., & Jöreskog, K. G. (1974). Intraclass reliability estimates: Testing structural assumptions. Educational and Psychological Measurement, 34, 25–33.
5. Graham, J. M. (2006). Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability What They Are and How to Use Them. Educational and Psychological Measurement, 66(6), 930–944. https://doi.org/10.1177/0013164406288165
6. Lucke, J. F. (2005). “Rassling the Hog”: The Influence of Correlated Item Error on Internal Consistency, Classical Reliability, and Congeneric Reliability. Applied Psychological Measurement, 29(2), 106–125. https://doi.org/10.1177/0146621604272739
7. Werts, C. E., Rock, D. R., Linn, R. L., & Jöreskog, K. G. (1978). A general method of estimating the reliability of a composite. Educational and Psychological Measurement, 38(4), 933–938. https://doi.org/10.1177/001316447803800412
8. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
9. Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2018). Multivariate data analysis (8th ed.). Cengage.
10. Padilla, M. (2019). A Primer on Reliability via Coefficient Alpha and Omega. Archives of Psychology, 3(8), Article 8. https://doi.org/10.31296/aop.v3i8.125
11. Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z
12. Bagozzi & Yi (1988), https://dx.doi.org/10.1177/009207038801600107
13. Guide & Ketokivi (2015), https://dx.doi.org/10.1016/S0272-6963(15)00056-X

External links

• RelCalc, tools to calculate congeneric reliability and other coefficients.
• Handbook of Management Scales, Wikibook that contains management related measurement models, their indicators and often congeneric reliability.