Two Voices on the Credibility Crisis in Psychology

Andrade, C. (2021). HARKing, Cherry-Picking, P-Hacking, Fishing Expeditions, and Data Dredging and Mining as Questionable Research Practices. The Journal of Clinical Psychiatry, 82(1), 20f13804. https://doi.org/10.4088/JCP.20f13804
Crossref   Google Scholar

Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The Rules of the Game Called Psychological Science. Perspectives on Psychological Science, 7(6), 543–554.
Crossref   Google Scholar

Bakker, M., Hartgerink, C. H., Wicherts, J. M., & van der Maas, H. L. (2016). Researchers’ Intuitions About Power in Psychological Research. Psychological Science, 27(8), 1069–1077. https://doi.org/10.1177/0956797616647519
Crossref   Google Scholar

Beer, R. D. (2020). Lost in words. Adaptive Behavior, 28(1), 19–21. https://doi.org/10.1177/1059712319867907
Crossref   Google Scholar

Bishop, D. (2019). Rein in the four horsemen of irreproducibility. Nature, 568(7753), 435. https://doi.org/10.1038/d41586-019-01307-2
Crossref   Google Scholar

Bishop, D. (2021). UBL & Elsevier seminars on Reproducible Research. YouTube https://www.youtube.com/watch?v=C-rk22as870&t=214s   Google Scholar

Borsboom, D., van der Maas, H. L. J., Dalege, J., Kievit, R. A., & Haig, B. D. (2021). Theory Construction Methodology: A Practical Framework for Building Theories in Psychology. Perspectives on Psychological Science, 16(4), 756–766. https://doi.org/10.1177/1745691620969647
Crossref   Google Scholar

Boyce, V., Mathur, M. B., & Frank, M. C. (2023, July 31). Eleven years of student replication projects provide evidence on the correlates of replicability in psychology. https://doi.org/10.31234/osf.io/dpyn6
Crossref   Google Scholar

Bringmann, L. F., Elmer, T., & Eronen, M. I. (2022). Back to Basics: The Importance of Conceptual Clarification in Psychological Science. Current Directions in Psychological Science, 31(4), 340–346. https://doi.org/10.1177/09637214221096485
Crossref   Google Scholar

Cohen, J. (1962). The statistical power of abnormal-social psychological research: a review. Journal of Abnormal Social Psychology, 65, 145–153.
Crossref   Google Scholar

Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312. https://doi.org/10.1037/0003-066X.45.12.1304
Crossref   Google Scholar

Cohen, J. (1994). The Earth is round (p < .05). American Psychologist, 49, 997–1003. https://doi.org/10.1037/0003-066X.49.12.997
Crossref   Google Scholar

Cumming, G. (2008). Replication and p Intervals: p Values Predict the Future Only Vaguely, but Confidence Intervals Do Much Better. Perspectives on Psychological Science, 3(4), 286–300. https://doi.org/10.1111/j.1745-6924.2008.00079.x
Crossref   Google Scholar

Cumming, G. (2014). The new statistics: why and how. Psychological Science, 25(1), 7–29.
Crossref   Google Scholar

de Groot, A. D. (1956/2014). The meaning of “significance” for different types of research (translation and annotated by E.-J. Wagenmakers, D. Borsboom, J. Verhagen, R. Kievit, M. Bakker, A. Cramer, D. Matzke, D. Mellenbergh, & H. L. J. van der Maas), Acta Psychologica, 148, 188–194. https://doi.org/10.1016/j.actpsy.2014.02.001
Crossref   Google Scholar

Eronen, M. I., & Bringmann, L. F. (2021). The Theory Crisis in Psychology: How to Move Forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/10.1177/1745691620970586
Crossref   Google Scholar

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://link.springer.com/article/10.3758/BF03193146
Crossref   Google Scholar

Fidler, F., Singleton Thorn, F., Barnett, A., Kambouris, S., & Kruger, A. (2018). The Epistemic Importance of Establishing the Absence of an Effect. Advances in Methods and Practices in Psychological Science, 1(2), 237–244. https://doi.org/10.1177/2515245918770407
Crossref   Google Scholar

Fisher, R. A. (1971). The design of experiments (9th edition). Hafner Press.   Google Scholar

Flake, J., & Fried, E. (2020). Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393
Crossref   Google Scholar

Fried, E., Flake, J., & Robinaugh, D. (2022). Revisiting the theoretical and methodological foundations of depression measurement. Nature Reviews Psychology, 1, 358–368. https://doi.org/10.1038/s44159-022-00050-2
Crossref   Google Scholar

Gelman, A., & Loken, E. (2014). The statistical crisis in science. American Scientist, 102, 460–465. https://doi.org/10.1511/2014.111.460
Crossref   Google Scholar

Haig, B. D. (2005). An Abductive Theory of Scientific Method. Psychological Methods, 10(4), 371–388. https://doi.org/10.1037/1082-989X.10.4.371
Crossref   Google Scholar

Haller, H., & Krauss, S. (2002). Misinterpretations of Significance: A Problem Students Share with Their Teachers? Methods of Psychological Research Online, 7(1).   Google Scholar

Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015). The fickle P value generates irreproducible results. Nature Methods, 12(3), 179–185. https://doi.org/10.1038/nmeth.3288
Crossref   Google Scholar

Haslbeck, J. M. B., Ryan, O., Robinaugh, D. J., Waldorp, L. J., & Borsboom, D. (2022). Modeling psychopathology: From data models to formal theories. Psychological Methods, 27(6), 930–957. https://doi.org/10.1037/met0000303
Crossref   Google Scholar

Heyard, R. (2022). Best practices in statistical design and reporting. University of Zurich. https://osf.io/t9rqm/   Google Scholar

Ioannidis, J. P. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
Crossref   Google Scholar

Kerr, N. L. (1998). HARKing: hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4
Crossref   Google Scholar

Lakens, D. (2023, July 24). Concerns about Replicability, Theorizing, Applicability, Generalizability, and Methodology across Two Crises in Social Psychology. https://doi.org/10.31234/osf.io/dtvs7
Crossref   Google Scholar

Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(4), 806–834. https://doi.org/10.1037/0022-006X.46.4.806
Crossref   Google Scholar

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1.
Crossref   Google Scholar

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. PNAS (Proceedings of the National Academy of Sciences of the United States of America), 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
Crossref   Google Scholar

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Struhl, M. K., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719–748. https://doi.org/10.1146/annurev-psych-020821-114157
Crossref   Google Scholar

Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/10.3758/s13423-019-01645-2
Crossref   Google Scholar

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), 1–8.
Crossref   Google Scholar

Robinaugh, D. J., Haslbeck, J. M. B., Ryan, O., Fried, E. I., & Waldorp, L. J. (2021). Invisible Hands and Fine Calipers: A Call to Use Formal Theory as a Toolkit for Theory Construction. Perspectives on Psychological Science, 16(4), 725–743. https://doi.org/10.1177/1745691620974697
Crossref   Google Scholar

Rossi, J. S. (1990). Statistical power of psychological research: What have we gained in 20 years. Journal of Consulting and Clinical Psychology, 58(5), 646.
Crossref   Google Scholar

Rozin, P. (2001). Social Psychology and Science: Some Lessons from Solomon Asch. Personality and Social Psychology Review, 5(1), 2–14. https://doi.org/10.1207/S15327957PSPR0501_1
Crossref   Google Scholar

Scheel, A. (2022). Why most psychological research findings are not even wrong. Infant and Child Development, 31(1): e2295. https://doi.org/10.1002/icd.2295
Crossref   Google Scholar

Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021). Why hypothesis testers should spend less time testing hypotheses. Perspectives on Psychological Science, 16(4), 744–755. https://doi.org/10.1177/1745691620966795
Crossref   Google Scholar

Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105(2), 309.
Crossref   Google Scholar

Smaldino, P. E., Calanchini, J., & Pickett, C. L. (2015). Theory development with agent-based models. Organizational Psychology Review, 5(4), 300–317. https://doi.org/10.1177/2041386614546944
Crossref   Google Scholar

Szollosi, A., & Donkin, C. (2021). Arrested Theory Development: The Misguided Distinction Between Exploratory and Confirmatory Research. Perspectives on Psychological Science, 16(4), 717–724. https://doi.org/10.1177/1745691620966796
Crossref   Google Scholar

van Dongen, N. N. N., van Bork, R., Finnemann, A., van der Maas, H., Robinaugh, D., Haslbeck, J. M. B., … Borsboom, D. (2022, April 13). Productive Explanation: A Framework for Evaluating Explanations in Psychological Science. https://doi.org/10.31234/osf.io/qd69g
Crossref   Google Scholar

van Rooij, I., Baggio, G. (2020). Theory development requires an epistemological sea change. Psychological Inquiry, 31(4), 321–325. https://doi.org/10.1080/1047840X.2020.1853477
Crossref   Google Scholar

Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J. Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., Matzke, D., Rouder, J. N., & Morey, R. D. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin & Review, 25(1), 35–57.
Crossref   Google Scholar

Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012). An Agenda for Purely Confirmatory Research. Perspectives on Psychological Science, 7(6), 632–638. https://doi.org/10.1177/1745691612463078
Crossref   Google Scholar

Wasserstein, R. (2015). ASA comment on a journal’s ban on null hypothesis statistical testing. Retrieved 05 Aug 2015, Sente.   Google Scholar

Westover, M. B., Westover, K. D., & Bianchi, M. T. (2011). Significance testing as perverse probabilistic reasoning. BMC Medicine, 9, 20. https://doi.org/10.1186/1741-7015-9-20
Crossref   Google Scholar

Wilkinson, L., APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594–604.
Crossref   Google Scholar

Wolski, P. (2016). Istotność statystyczna II. Pułapki interpretacyjne [Statistical significance II. Interpretive pitfalls]. Rocznik Kognitywistyczny [Yearbook of Cognitive Science], 9, 59–70 (in Polish). https://doi.org/10.4467/20843895RK.16.006.6412
Crossref   Google Scholar

Woolston, C. (2015). Psychology journal bans P values. Nature, 519(7541), 9.
Crossref   Google Scholar

Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, E1. https://doi.org/10.1017/S0140525X20001685
Crossref   Google Scholar