Dwa głosy o kryzysie wiarygodności w psychologii

Arkadiusz Białek

Uniwersytet Jagielloński, Instytut Psychologii
https://orcid.org/0000-0002-9002-4764

Piotr Wolski

Uniwersytet Jagielloński, Instytut Psychologii
https://orcid.org/0000-0002-7028-6142


Abstract

Choć różne niedociągnięcia i wady sposobu prowadzenia badań i analizowania wyników w psychologii oraz innych naukach społecznych dostrzegano już dawno, ostatnie lata wyróżnia zarówno powszechność, jak i zakres tej krytyki. Pojawia się też więcej propozycji naprawy. W artykule skupiamy się na wybranych, kluczowych naszym zdaniem, obszarach kryzysu wiarygodności w psychologii. Piotr Wolski omawia te, które wiążą się z niewłaściwym rozumieniem i stosowaniem testów istotności, Arkadiusz Białek charakteryzuje niektóre z obniżających wiarygodność badań psychologicznych niewłaściwych praktyk badawczych oraz pokazuje, jak można im przeciwdziałać. Choć stosowanie dobrych praktyk badawczych może poprawić reprodukowalność i replikowalność wyników badań, to postulowana reforma powinna objąć swoim zakresem także sposób tworzenia teorii. Omawiana propozycja zasad tworzenia teorii w psychologii prowadzi do serii praktyczych kroków. W przeciwieństwie do dotąd dominującego medelu hipototeczno-dedukcyjnego za punkt wyjścia przyjmuje się identyfikację i opis fenomenu. Sformułowane poprzez abdukcję wyjaśnienie fenomenu jest następnie formalizowane w równaniach matematycznych lub symulacjach komputerowych i weryfikowane. Przestrzeganie dobrych praktyk badawczych oraz poprawne tworzenie teorii ma szansę dostarczyć psychologii bardziej solidnych podstaw i uczynić ją nauką o kumulatywnym charakterze.  


Keywords:

krzyzys wiarygodności, wnioskowanie statystyczne, wartość p, testy istotności, niewłaściwe praktyki badawcze, tworzenie teorii


Andrade, C. (2021). HARKing, Cherry-Picking, P-Hacking, Fishing Expeditions, and Data Dredging and Mining as Questionable Research Practices. The Journal of Clinical Psychiatry, 82(1), 20f13804. https://doi.org/10.4088/JCP.20f13804
Crossref   Google Scholar

Bakker, M., van Dijk, A., Wicherts, J. M. (2012). The Rules of the Game Called Psychological Science. Perspectives on Psychological Science, 7(6), 543–554.
Crossref   Google Scholar

Bakker, M., Hartgerink, C. H., Wicherts, J. M., van der Maas, H. L. (2016). Researchers’ Intuitions About Power in Psychological Research. Psychological Science, 27(8), 1069–1077. https://doi.org/10.1177/0956797616647519
Crossref   Google Scholar

Beer, R. D. (2020). Lost in words. Adaptive Behavior, 28(1), 19–21. https://doi.org/10.1177/1059712319867907
Crossref   Google Scholar

Bishop, D. (2019). Rein in the four horsemen of irreproducibility. Nature, 568(7753), 435. https://doi.org/10.1038/d41586-019-01307-2
Crossref   Google Scholar

Bishop, D. (2021). UBL & Elsevier seminars on Reproducible Research. YouTube https://www.youtube.com/watch?v=C-rk22as870&t=214s   Google Scholar

Borsboom, D., van der Maas, H. L. J., Dalege, J., Kievit, R. A., Haig, B. D. (2021). Theory Construction Methodology: A Practical Framework for Building Theories in Psychology. Perspectives on Psychological Science, 16(4), 756–766. https://doi.org/10.1177/1745691620969647
Crossref   Google Scholar

Boyce, V., Mathur, M. B., Frank, M. C. (2023, July 31). Eleven years of student replication projects provide evidence on the correlates of replicability in psychology. https://doi.org/10.31234/osf.io/dpyn6
Crossref   Google Scholar

Bringmann, L. F., Elmer, T., Eronen, M. I. (2022). Back to Basics: The Importance of Conceptual Clarification in Psychological Science. Current Directions in Psychological Science, 31(4), 340–346. https://doi.org/10.1177/09637214221096485
Crossref   Google Scholar

Cohen, J. (1962). The statistical power of abnormal-social psychological research: a review. Journal of Abnormal Social Psychology, 65, 145–153.
Crossref   Google Scholar

Cohen, J. (1990/2006). Things I have learned (so far). American Psychologist, 45, 1304–1312. https://doi.org/10.1037/0003-066X.45.12.1304 (tłum. Cohen, J. (2006). O tym, czego się nauczyłem (jak dotąd). W: J. Brzeziński i J. Siuta (red.), Metodologiczne i statystyczne problemy psychologii (s. 75–99). Wydawnictwo Zysk i S-ka.)
Crossref   Google Scholar

Cohen, J. (1994/2006). The Earth is round (p < .05). American Psychologist, 49, 997–1003. https://doi.org/10.1037/0003-066X.49.12.997 (tłum. Cohen, J. (2006). Ziemia jest okrągła (p < 0,05). W: J. Brzeziński i J. Siuta (red.), Metodologiczne i statystyczne problemy psychologii (s. 100–118). Wydawnictwo Zysk i S-ka.)
Crossref   Google Scholar

Cumming, G. (2008). Replication and p Intervals: p Values Predict the Future Only Vaguely, but Confidence Intervals Do Much Better. Perspectives on Psychological Science, 3(4), 286–300. https://doi.org/10.1111/j.1745-6924.2008.00079.x
Crossref   Google Scholar

Cumming, G. (2014). The new statistics: why and how. Psychological Science, 25(1), 7–29.
Crossref   Google Scholar

de Groot, A. D. (1956/2014). The meaning of “significance” for different types of research (tłum. i adnotacje E.-J. Wagenmakers, D. Borsboom, J. Verhagen, R. Kievit, M. Bakker, A. Cramer, D. Matzke, D. Mellenbergh, H. L. J. van der Maas), Acta Psychologica, 148, 188–194. https://doi.org/10.1016/j.actpsy.2014.02.001
Crossref   Google Scholar

Eronen, M. I., Bringmann, L. F. (2021). The Theory Crisis in Psychology: How to Move Forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/10.1177/1745691620970586
Crossref   Google Scholar

Faul, F., Erdfelder, E., Lang, A.-G., Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://link.springer.com/article/10.3758/BF03193146
Crossref   Google Scholar

Fidler, F., Singleton Thorn, F., Barnett, A., Kambouris, S., Kruger, A. (2018). The Epistemic Importance of Establishing the Absence of an Effect. Advances in Methods and Practices in Psychological Science, 1(2), 237–244. https://doi.org/10.1177/2515245918770407
Crossref   Google Scholar

Fisher, R. A. (1971). The design of experiments (wyd. 9). Hafner Press.   Google Scholar

Flake, J., Fried, E. (2020). Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393
Crossref   Google Scholar

Fried, E., Flake, J., Robinaugh, D. (2022). Revisiting the theoretical and methodological foundations of depression measurement. Nature Reviews Psychology, 1, 358–368. https://doi.org/10.1038/s44159-022-00050-2
Crossref   Google Scholar

Gelman, A., Loken, E. (2014). The statistical crisis in science. American Scientist, 102, 460–465. https://doi.org/10.1511/2014.111.460
Crossref   Google Scholar

Haig, B. D. (2005). An Abductive Theory of Scientific Method. Psychological Methods, 10(4), 371–388. https://doi.org/10.1037/1082-989X.10.4.371
Crossref   Google Scholar

Haller, H., Krauss, S. (2002). Misinterpretations of Significance: A Problem Students Share with Their Teachers? Methods of Psychological Research Online, 7(1).   Google Scholar

Halsey, L. G., Curran-Everett, D., Vowler, S. L., Drummond, G. B. (2015). The fickle P value generates irreproducible results. Nature Methods, 12(3), 179–185. https://doi.org/10.1038/nmeth.3288
Crossref   Google Scholar

Haslbeck, J. M. B., Ryan, O., Robinaugh, D. J., Waldorp, L. J., Borsboom, D. (2022). Modeling psychopathology: From data models to formal theories. Psychological Methods, 27(6), 930–957. https://doi.org/10.1037/met0000303
Crossref   Google Scholar

Heyard, R. (2022). Best practices in statistical design and reporting. University of Zurich. https://osf.io/t9rqm/   Google Scholar

Ioannidis, J. P. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
Crossref   Google Scholar

Kerr, N. L. (1998). HARKing: hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4
Crossref   Google Scholar

Lakens, D. (2023, July 24). Concerns about Replicability, Theorizing, Applicability, Generalizability, and Methodology across Two Crises in Social Psychology. https://doi.org/10.31234/osf.io/dtvs7
Crossref   Google Scholar

Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(4), 806–834. https://doi.org/10.1037/0022-006X.46.4.806
Crossref   Google Scholar

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1.
Crossref   Google Scholar

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., Mellor, D. T. (2018). The preregistration revolution. PNAS (Proceedings of the National Academy of Sciences of the United States of America), 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
Crossref   Google Scholar

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Struhl, M. K., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719–748. https://doi.org/10.1146/annurev-psych-020821-114157
Crossref   Google Scholar

Oberauer, K., Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/10.3758/s13423-019-01645-2
Crossref   Google Scholar

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), 1–8.
Crossref   Google Scholar

Robinaugh, D. J., Haslbeck, J. M. B., Ryan, O., Fried, E. I., Waldorp, L. J. (2021). Invisible Hands and Fine Calipers: A Call to Use Formal Theory as a Toolkit for Theory Construction. Perspectives on Psychological Science, 16(4), 725–743. https://doi.org/10.1177/1745691620974697
Crossref   Google Scholar

Rossi, J. S. (1990). Statistical power of psychological research: What have we gained in 20 years. Journal of Consulting and Clinical Psychology, 58(5), 646.
Crossref   Google Scholar

Rozin, P. (2001). Social Psychology and Science: Some Lessons from Solomon Asch. Personality and Social Psychology Review, 5(1), 2–14. https://doi.org/10.1207/S15327957PSPR0501_1
Crossref   Google Scholar

Scheel, A. (2022). Why most psychological research findings are not even wrong. Infant and Child Development, 31(1): e2295. https://doi.org/10.1002/icd.2295
Crossref   Google Scholar

Scheel, A. M., Tiokhin, L., Isager, P. M., Lakens, D. (2021). Why hypothesis testers should spend less time testing hypotheses. Perspectives on Psychological Science, 16(4), 744–755. https://doi.org/10.1177/1745691620966795
Crossref   Google Scholar

Sedlmeier, P., Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105(2), 309.
Crossref   Google Scholar

SJP. (b.d.). Słownik języka polskiego PWN online (https://sjp.pwn.pl/).   Google Scholar

Smaldino, P. E., Calanchini, J., Pickett, C. L. (2015). Theory development with agent-based models. Organizational Psychology Review, 5(4), 300–317. https://doi.org/10.1177/2041386614546944
Crossref   Google Scholar

Szollosi, A., Donkin, C. (2021). Arrested Theory Development: The Misguided Distinction Between Exploratory and Confirmatory Research. Perspectives on Psychological Science, 16(4), 717–724. https://doi.org/10.1177/1745691620966796
Crossref   Google Scholar

van Dongen, N. N. N., van Bork, R., Finnemann, A., van der Maas, H., Robinaugh, D., Haslbeck, J. M. B., […] Borsboom, D. (2022, April 13). Productive Explanation: A Framework for Evaluating Explanations in Psychological Science. https://doi.org/10.31234/osf.io/qd69g
Crossref   Google Scholar

van Rooij, I., Baggio, G. (2020). Theory development requires an epistemological sea change. Psychological Inquiry, 31(4), 321–325. https://doi.org/10.1080/1047840X.2020.1853477
Crossref   Google Scholar

Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J. Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., Matzke, D., Rouder, J. N., Morey, R. D. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin & Review, 25(1), 35–57.
Crossref   Google Scholar

Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., Kievit, R. A. (2012). An Agenda for Purely Confirmatory Research. Perspectives on Psychological Science, 7(6), 632–638. https://doi.org/10.1177/1745691612463078
Crossref   Google Scholar

Wasserstein, R. (2015). ASA comment on a journal’s ban on null hypothesis statistical testing. Retrieved 05 Aug 2015, Sente.   Google Scholar

Westover, M. B., Westover, K. D., Bianchi, M. T. (2011). Significance testing as perverse probabilistic reasoning. BMC Medicine, 9, 20. https://doi.org/10.1186/1741-7015-9-20
Crossref   Google Scholar

Wilkinson, L., APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594–604.
Crossref   Google Scholar

Wolski, P. (2016). Istotność statystyczna II. Pułapki interpretacyjne. Rocznik Kognitywistyczny, 9, 59–70. https://doi.org/10.4467/20843895RK.16.006.6412
Crossref   Google Scholar

Woolston, C. (2015). Psychology journal bans P values. Nature, 519(7541), 9.
Crossref   Google Scholar

Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, E1. https://doi.org/10.1017/S0140525X20001685
Crossref   Google Scholar


Published
2023-10-26

Cited by

Białek, A., & Wolski, P. (2023). Dwa głosy o kryzysie wiarygodności w psychologii. The Review of Psychology, 66(1), 9–26. https://doi.org/10.31648/przegldpsychologiczny.9455

Arkadiusz Białek 
Uniwersytet Jagielloński, Instytut Psychologii
https://orcid.org/0000-0002-9002-4764
Piotr Wolski 
Uniwersytet Jagielloński, Instytut Psychologii
https://orcid.org/0000-0002-7028-6142