Cronbach’s alpha - what makes it really good? Some advice for planning and criticizing psychological questionnaires

Anselmi, P., Colledani, D., & Robusto, E. (2019). A comparison of classical and modern measures of internal consistency. Frontiers in Psychology, 10, Article 2714. https://doi.org/10.3389/fpsyg.2019.02714   Google Scholar

Bajpai, S., & Bajpai, R. (2014). Goodness of measurement: Reliability and validity. International Journal of Medical Science and Public Health, 3(2), 112‒115. https://doi.org/10.5455/ijmsph.2013.191120133   Google Scholar

Barbaranelli, C., Lee, C. S., Vellone, E., & Riegel, B. (2015). The problem with Cronbach's alpha: comment on Sijtsma and van der Ark (2015). Nursing Research, 64(2), 140‒145. https://doi.org/10.1097/NNR.0000000000000079   Google Scholar

Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137‒143. https://doi.org/10.1007/s11336-008-9100-1   Google Scholar

Bland, J. M., & Altman, D. G. (1997). Statistics notes: Cronbach's alpha. British Medical Journal, 314(7080), Article 572. https://doi.org/10.1136/bmj.314.7080.572   Google Scholar

Bonett, D. G., & Wright, T. A. (2015). Cronbach's alpha reliability: Interval estimation, hypothesis testing, and sample size planning. Journal of Organizational Behavior, 36(1), 3‒15. https://doi.org/10.1002/job.1960   Google Scholar

Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71(3), 425‒440. https://doi.org/10.1007/s11336-006-1447-6   Google Scholar

Borsboom, D., & Mellenbergh, G. J. (2002). True scores, latent variables, and constructs: A comment on Schmidt and Hunter. Intelligence, 30(6), 505‒514. https://doi.org/10.1016/S0160-2896(02)00082-X   Google Scholar

Brannick, M. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16(3), 201‒213. https://doi.org/10.1002/job.4030160303   Google Scholar

Breckler, S. J. (1990). Applications of covariance structure modeling in psychology: Cause for concern? Psychological Bulletin, 107(2), 260‒273. https://doi.org/10.1037/0033-2909.107.2.260   Google Scholar

Bujang, M. A., Omar, E. D., & Baharum, N. A. (2018). A review on sample size determination for Cronbach’s alpha test: a simple guide for researchers. The Malaysian Journal Of Medical Sciences: MJMS, 25(6), 85‒99. https://doi.org/10.21315/mjms2018.25.6.9   Google Scholar

Chan, E. K. (2014). Standards and guidelines for validation practices: Development and evaluation of measurement instruments. In Validity and validation in social, behavioral, and health sciences (pp. 9‒24). Springer.   Google Scholar

Chang, L. (1994). A psychometric evaluation of 4-point and 6-point Likert-type scales in relation to reliability and validity. Applied Psychological Measurement, 18(3), 205‒215. https://doi.org/10.1177/014662169401800302   Google Scholar

Charter, R. A. (1999). Sample size requirements for precise estimates of reliability, generalizability, and validity coefficients. Journal of Clinical and Experimental Neuropsychology, 21(4), 559‒566. https://doi.org/10.1007/978-3-319-07794-9_2   Google Scholar

Cho, E. (2022). The accuracy of reliability coefficients: A reanalysis of existing simulations. Psychological Methods. Online first. https://doi.org/10.1037/met0000475   Google Scholar

Cho, E., & Kim, S. (2015). Cronbach’s coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207‒230. https://doi.org/10.1177/1094428114555994   Google Scholar

Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Erlbaum.   Google Scholar

de Vet, H. C., Mokkink, L. B., Mosmuller, D. G., & Terwee, C. B. (2017). Spearman–Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applications. Journal of Clinical Epidemiology, 85, 45‒49. https://doi.org/10.1016/j.jclinepi.2017.01.013   Google Scholar

DeVellis, R. F. (2006). Classical test theory. Medical Care, 44(11), S50‒S59. https://doi.org/10.1097/01.mlr.0000245426.10853.30   Google Scholar

Dimov, I. T. (2008). Monte Carlo methods for applied scientists. World Scientific. https://doi.org/10.1142/9789812779892   Google Scholar

Duhachek, A., Coughlan, A. T., & Iacobucci, D. (2005). Results on the standard error of the coefficient alpha index of reliability. Marketing Science, 24(2), 294‒301. https://doi.org/10.1287/mksc.1040.0097   Google Scholar

Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399‒412. https://doi.org/10.1111/bjop.12046   Google Scholar

Dunn, W. L., & Shultis, J. K. (2011). Exploring Monte Carlo methods. Elsevier. https://doi.org/10.1016/B978-0-444-51575-9.00007-5   Google Scholar

Eisinga, R., Grotenhuis, M. T., & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? International Journal of Public Health, 58(4), 637‒642. https://doi.org/10.1007/s00038-012-0416-3   Google Scholar

Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370‒378. https://doi.org/10.1177/1948550617693063   Google Scholar

Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484‒501. https://doi.org/10.1177/2515245920951747   Google Scholar

Gignac, G. E., Bates, T. C., & Jang, K. L. (2007). Implications relevant to CFA model misfit, reliability, and the five-factor model as measured by the NEO-FFI. Personality and Individual Differences, 43(5), 1051‒1062. https://doi.org/10.1016/j.paid.2007.02.024   Google Scholar

Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The Qualitative Report, 8(4), 597‒607.   Google Scholar

Green, S. B., & Yang, Y. (2009). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74(1), 121‒135. https://doi.org/10.1007/s11336-008-9098-4   Google Scholar

Gruszczyńska, E. (2012). Kwestionariusz Samooceny Zysków i Strat – polska adaptacja Cor-Evaluation Se Hobfolla i jej podstawowe właściwości psychometryczne [Profit and Loss Self-Assessment Questionnaire – Polish adaptation of Hobfoll's Cor-Evaluation Se and its basic psychometric properties]. In E. Bielawska-Batorowicz & B. Dudek (Eds.), Teoria zachowania zasobow Stevana E. Hobfolla. Polskie doświadczenia [Stevan E. Hobfoll's theory of conservation of resources. Polish experience]. Wydawnictwo Uniwersytetu Łódzkiego.   Google Scholar

Guidroz, A. M., Yankelevich, M., Barger, P., Gillespie, M. A., & Zickar, M. J. (2009). Practical considerations for creating and using organizational survey norms: Lessons from two long-term projects. Consulting Psychology Journal: Practice and Research, 61(2), 85‒102. https://doi.org/10.1037/a0015969   Google Scholar

Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But… Communication Methods and Measures, 14(1), 1‒24. https://doi.org/10.1080/19312458.2020.1718629   Google Scholar

Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34(3), 177‒189. https://doi.org/10.1080/07481756.2002.12069034   Google Scholar

Hoyt, W. T., Warbasse, R. E., & Chu, E. Y. (2006). Construct validation in counseling psychology research. The Counseling Psychologist, 34(6), 769‒805. https://doi.org/10.1177/0011000006287389   Google Scholar

Jolliffe, I. T., & Cadima, J. (2016). Principal component analysis: a review and recent developments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2065), Article 20150202. https://doi.org/10.1098/rsta.2015.0202   Google Scholar

Kalkbrenner, M. T. (2023). Alpha, Omega, and H internal consistency reliability estimates: Reviewing these options and when to use them. Counseling Outcome Research and Evaluation, 14(1), 77‒88. https://doi.org/10.1080/21501378.2021.1940118   Google Scholar

Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1‒73. https://doi.org/10.1111/jedm.12000   Google Scholar

Leung, S. O. (2011). A comparison of psychometric properties and normality in 4-, 5-, 6-, and 11-point Likert scales. Journal of Social Service Research, 37(4), 412‒421. https://doi.org/10.1080/01488376.2011.580697   Google Scholar

Li, H., Rosenthal, R., & Rubin, D. B. (1996). Reliability of measurement in psychology: From Spearman-Brown to maximal reliability. Psychological Methods, 1(1), 98‒107. https://doi.org/10.1037/1082-989X.1.1.98   Google Scholar

Lucke, J. F. (2005). The α and the ω of congeneric test theory: An extension of reliability and internal consistency to heterogeneous tests. Applied Psychological Measurement, 29(1), 65‒81. https://doi.org/10.1177/0146621604270882   Google Scholar

Macey, W. H., & Eldridge, L. D. (2006). National norms versus consortium data: What do they tell us. In A. I. Kraut (Ed.), Getting action from organizational surveys: New concepts, technologies, and applications (pp. 352‒376). Jossey-Bass.   Google Scholar

Matell, M. S., & Jacoby, J. (1972). Is there an optimal number of alternatives for Likert-scale items? Effects of testing time and scale properties. Journal of Applied Psychology, 56(6), 506‒509. https://doi.org/10.1037/h0033601   Google Scholar

McCrae, R. R., Kurtz, J. E., Yamagata, S., & Terracciano, A. (2011). Internal consistency, retest reliability, and their implications for personality scale validity. Personality and Social Psychology Review, 15(1), 28‒50. https://doi.org/10.1177/1088868310366253   Google Scholar

McDonald, R. P. (2013). Test theory: A unified treatment. Psychology Press. https://doi.org/10.4324/9781410601087   Google Scholar

McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412‒433. https://doi.org/10.1037/met0000144   Google Scholar

Metsämuuronen, J. (2022). The effect of various simultaneous sources of mechanical error in the estimators of correlation causing deflation in reliability: Seeking the best options of correlation for deflation-corrected reliability. Behaviormetrika, 49(1), 91‒130. https://doi.org/10.1007/s41237-022-00158-y   Google Scholar

Pastore, M., & Lombardi, L. (2014). The impact of faking on Cronbach’s alpha for dichotomous and ordered rating scores. Quality & Quantity, 48(3), 1191‒1211. https://doi.org/10.1007/s11135-013-9829-1   Google Scholar

Ponterotto, J. G., & Ruckdeschel, D. E. (2007). An overview of coefficient alpha and a reliability matrix for estimating adequacy of internal consistency coefficients with psychological research measures. Perceptual and Motor Skills, 105(3), 997‒1014. https://doi.org/10.2466/pms.105.3.997-1014   Google Scholar

Preston, C. C., & Colman, A. M. (2000). Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104(1), 1‒15. https://doi.org/10.1016/S0001-6918(99)00050-5   Google Scholar

Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge. https://doi.org/10.4324/9780203841624   Google Scholar

Revelle, W., & Condon, D. M. (2019). Reliability from α to ω: A tutorial. Psychological Assessment, 31(12), 1395‒1411. https://doi.org/10.1037/pas0000754   Google Scholar

Šerbetar, I., & Sedlar, I. (2016). Assessing reliability of a multi-dimensional scale by coefficient alpha. Journal of Elementary Education, 9(1/2), 189‒196.   Google Scholar

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107‒120. https://doi.org/10.1007/s11336-008-9101-0   Google Scholar

Sijtsma, K. (2020). Measurement Models for Psychological Attributes: Classical Test Theory, Factor Analysis, Item Response Theory, and Latent Class Models. CRC Press. https://doi.org/10.1201/9780429112447-2   Google Scholar

Sijtsma, K., & Pfadt, J. M. (2021). Part II: On the use, the misuse, and the very limited usefulness of Cronbach’s alpha: Discussing lower bounds and correlated errors. Psychometrika, 86(4), 843‒860. https://doi.org/10.1007/s11336-021-09789-8   Google Scholar

Streiner, D. L. (2003). Starting at the beginning: an introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99‒103. https://doi.org/10.1207/S15327752JPA8001_18   Google Scholar

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Allyn & Bacon.   Google Scholar

Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273‒1296. https://doi.org/10.1007/s11165-016-9602-2   Google Scholar

Taherdoost, H. (2022). What is the best response scale for survey and questionnaire design; review of different lengths of rating scale / attitude scale / Likert scale. International Journal of Academic Research in Management, 8(1), 1‒10.   Google Scholar

Tang, W., Cui, Y., & Babenko, O. (2014). Internal consistency: Do we really know what it is and how to assess it. Journal of Psychology and Behavioral Science, 2(2), 205‒220.   Google Scholar

Ten Berge, J. M., & Sočan, G. (2004). The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69(4), 613‒625. https://doi.org/10.1007/BF02289858   Google Scholar

Thall, P. F., & Vail, S. C. (1990). Some covariance models for longitudinal count data with overdispersion. Biometrics, 46(3), 657‒671. https://doi.org/10.2307/2532086   Google Scholar

Thigpen, N. N., Kappenman, E. S., & Keil, A. (2017). Assessing the internal consistency of the event‐related potential: An example analysis. Psychophysiology, 54(1), 123‒138. https://doi.org/10.1111/psyp.12629   Google Scholar

Thompson, B. (2002). Score reliability: Contemporary thinking on reliability issues (1st ed.). Sage Publications, Inc. https://doi.org/10.4135/9781412985789.n1   Google Scholar

Trizano-Hermosilla, I., & Alvarado, J. M. (2016). Best alternatives to Cronbach's alpha reliability in realistic conditions: congeneric and asymmetrical measurements. Frontiers in Psychology, 7, Article 769. https://doi.org/10.3389/fpsyg.2016.00769   Google Scholar

Vaske, J. J., Beaman, J., & Sponarski, C. C. (2017). Rethinking internal consistency in Cronbach's alpha. Leisure Sciences, 39(2), 163‒173. https://doi.org/10.1080/01490400.2015.1127189   Google Scholar

Vehkalahti, K., Puntanen, S., Tarkkonen, L. (2006). Estimation of reliability: a better alternative for Cronbach's alpha. Department of Mathematics and Statistics, University of Helsinki.   Google Scholar

Zumbo, B. D., & Chan, E. K. (2014). Validity and validation in social, behavioral, and health sciences. Social Indicators Research Series, Vol. 54. Springer. https://doi.org/10.1007/978-3-319-07794-9   Google Scholar