Cognitive Metascience: A New Approach to the Study of Theories

Marcin Miłkowski

Polish Academy of Sciences, Institute of Philosophy and Sociology
https://orcid.org/0000-0001-7646-5742


Abstrakt

In light of the recent credibility crisis in psychology, this paper argues for a greater emphasis on theorizing in scientific research. Although reliable experimental evidence, preregistration, methodological rigor, and new computational frameworks for modeling are important, scientific progress also relies on properly functioning theories. However, the current understanding of the role of theorizing in psychology is lacking, which may lead to future crises. Theories should not be viewed as mere speculations or simple inductive generalizations.

To address this issue, the author introduces a framework called “cognitive metascience,” which studies the processes and results of evaluating scientific practice. This study should proceed both qualitatively, as in traditional science and technology studies and cognitive science, and quantitatively, by analyzing scientific discourse using language technology.

By analyzing theories as cognitive artifacts that support cognitive tasks, this paper aims to shed more light on their nature. This perspective reveals that multiple distinct theories serve entirely different roles, and studying these roles, along with their epistemic vices and virtues, can provide insight into how theorizing should proceed. The author urges a change in research culture to appreciate the variety of distinct theories and to systematically advance scientific progress. 


Słowa kluczowe:

theory crisis, cognitive metascience, cognitive artifact, theoretical virtue, epistemic criteria


Afeltowicz, Ł., & Wachowski, W. (2015). How Far we Can Go Without Looking Under the Skin: The Bounds of Cognitive Science. Studies in Logic, Grammar and Rhetoric, 40(1), 91–109. https://doi.org/ 10.1515/slgr-2015-0005
Crossref   Google Scholar

Almaatouq, A., Griffiths, T. L., Suchow, J. W., Whiting, M. E., Evans, J., & Watts, D. J. (2022). Beyond Playing 20 Questions with Nature: Integrative Experiment Design in the Social and Behavioral Sciences. Behavioral and Brain Sciences, 1–55. https://doi.org/ 10.1017/S0140525X22002874
Crossref   Google Scholar

American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). American Psychiatric Association.   Google Scholar

Anderson, J. R. (2007). How Can the Mind Occur in the Physical Universe? Oxford University Press.
Crossref   Google Scholar

Aronova, E., Oertzen, C. von, & Sepkoski, D. (Eds.). (2017). Data histories. University of Chicago Press.   Google Scholar

Autzen, B. (2021). Is the replication crisis a base-rate fallacy? Theoretical Medicine and Bioethics, 42(5), 233–243. https://doi.org/ 10.1007/s11017-022-09561-8
Crossref   Google Scholar

Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature News, 533(7604), 452. https://doi.org/ 10.1038/533452a
Crossref   Google Scholar

Baumeister, R. F., & Tice, D. M. (2022). Ego Depletion is the Best Replicated Finding in All of Social Psychology. Scholarly Journal of Psychology and Behavioral Sciences, 6(2), 686–688. https://doi.org/ 10.32474/SJPBS.2021.06.000234   Google Scholar

Bird, A. (2021). Understanding the Replication Crisis as a Base Rate Fallacy. The British Journal for the Philosophy of Science, 72(4), 965–993. https://doi.org/ 10.1093/bjps/axy051
Crossref   Google Scholar

Boekel, W., Wagenmakers, E.-J., Belay, L., Verhagen, J., Brown, S., & Forstmann, B. U. (2015). A purely confirmatory replication study of structural brain-behavior correlations. Cortex, 66, 115–133. https://doi.org/ 10.1016/j.cortex.2014.11.019
Crossref   Google Scholar

Bogen, J., & Woodward, J. (1988). Saving the Phenomena. The Philosophical Review, 97(3), 303. https://doi.org/ 10.2307/2185445
Crossref   Google Scholar

Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64(9), 1089–1108. https://doi.org/ 10.1002/jclp.20503
Crossref   Google Scholar

Borsboom, D., Cramer, A., & Kalis, A. (2018). Brain disorders? Not really… Why network structures block reductionism in psychopathology research. Behavioral and Brain Sciences, 1–54. https://doi.org/ 10.1017/S0140525X17002266
Crossref   Google Scholar

Borsboom, D., & Cramer, A. O. J. (2013). Network Analysis: An Integrative Approach to the Structure of Psychopathology. Annual Review of Clinical Psychology, 9(1), 91–121. https://doi.org/ 10.1146/annurev-clinpsy-050212-185608
Crossref   Google Scholar

Botvinik-Nezer, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber, J., Johannesson, M., … Schonberg, T. (2020). Variability in the analysis of a single neuroimaging dataset by many teams. Nature, 582(7810), 84–88. https://doi.org/ 10.1038/s41586-020-2314-9   Google Scholar

Bower, G. H. (1993). The fragmentation of psychology? American Psychologist, 48(8), 905–907. (1994-00003-001). https://doi.org/ 10.1037/0003-066X.48.8.905
Crossref   Google Scholar

Bringmann, L. F., & Eronen, M. Ilkka. (2016). Heating up the measurement debate: What psychologists can learn from the history of physics. Theory & Psychology, 26(1), 27–43. https://doi.org/ 10.1177/0959354315617253
Crossref   Google Scholar

Broadbent, A. (2018). Prediction, Understanding, and Medicine. The Journal of Medicine and Philosophy: A Forum for Bioethics and Philosophy of Medicine, 43(3), 289–305. https://doi.org/ 10.1093/jmp/jhy003
Crossref   Google Scholar

Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365. https://doi.org/ 10.1038/nrn3475
Crossref   Google Scholar

Cahalan, S. (2019). The Great Pretender. Grand Central Publishers.   Google Scholar

Callebaut, W. (1993). Taking the naturalistic turn or how real philosophy of science is done. University of Chicago Press.   Google Scholar

Callebaut, W. (2013). Naturalizing Theorizing: Beyond a Theory of Biological Theories. Biological Theory, 7(4), 413–429. https://doi.org/ 10.1007/s13752-013-0122-2
Crossref   Google Scholar

Carsel, T., Demos, A. P., & Motyl, M. (2018). Strong scientific theorizing is needed to improve replicability in psychological science. Behavioral and Brain Sciences, 41, e123. https://doi.org/ 10.1017/S0140525X1800078X
Crossref   Google Scholar

Chang, H. (2017). VI—Operational Coherence as the Source of Truth. Proceedings of the Aristotelian Society, 117(2), 103–122. https://doi.org/ 10.1093/arisoc/aox004
Crossref   Google Scholar

Chomsky, N. (1959). Review of Verbal Behavior by B. F. Skinner. Language, 35(1), 26–58.
Crossref   Google Scholar

Cooper, R. P., & Shallice, T. (1995). Soar and the case for unified theories of cognition. Cognition, 55(2), 115–149. https://doi.org/ 10.1016/0010-0277(94)00644-Z
Crossref   Google Scholar

Cowan, N., Belletier, C., Doherty, J. M., Jaroslawska, A. J., Rhodes, S., Forsberg, A., … Logie, R. H. (2020). How Do Scientific Views Change? Notes From an Extended Adversarial Collaboration. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 15(4), 1011–1025. https://doi.org/ 10.1177/1745691620906415
Crossref   Google Scholar

Craver, C. F. (2007). Explaining the Brain: Mechanisms and the Mosaic Unity of Neuroscience. Oxford University Press.
Crossref   Google Scholar

Craver, C. F. (2009). Mechanisms and natural kinds. Philosophical Psychology, 22(5), 575–594. https://doi.org/ 10.1080/09515080903238930
Crossref   Google Scholar

Cummins, R. (2000). “How does it work” versus “what are the laws?”: Two conceptions of psychological explanation. In F. Keil & R. A. Wilson (Eds.), Explanation and Cognition (pp. 117–145). MIT Press.
Crossref   Google Scholar

Dale, R., Dietrich, E., & Chemero, A. (2009). Explanatory Pluralism in Cognitive Science. Cognitive Science, 33(5), 739–742. https://doi.org/ 10.1111/j.1551-6709.2009.01042.x
Crossref   Google Scholar

Dang, J. (2016). Commentary: A Multilab Preregistered Replication of the Ego-Depletion Effect. Frontiers in Psychology, 7, 1155. https://doi.org/ 10.3389/fpsyg.2016.01155
Crossref   Google Scholar

Del Pin, S. H., Skóra, Z., Sandberg, K., Overgaard, M., & Wierzchoń, M. (2021). Comparing theories of consciousness: Why it matters and how to do it. Neuroscience of Consciousness, 2021(2), niab019. https://doi.org/ 10.1093/nc/niab019
Crossref   Google Scholar

Dellsén, F. (2020). The epistemic impact of theorizing: Generation bias implies evaluation bias. Philosophical Studies, 177(12), 3661–3678. https://doi.org/ 10.1007/s11098-019-01387-w
Crossref   Google Scholar

Di Nardo, P. A., O’Brien, G. T., Barlow, D. H., Waddell, M. T., & Blanchard, E. B. (1983). Reliability of DSM-III Anxiety Disorder Categories Using a New Structured Interview. Archives of General Psychiatry, 40(10), 1070–1074. https://doi.org/ 10.1001/archpsyc.1983.01790090032005
Crossref   Google Scholar

Dror, I. E., & Gallogly, D. P. (1999). Computational analyses in cognitive neuroscience: In defense of biological implausibility. Psychonomic Bulletin & Review, 6(2), 173–182. https://doi.org/ 10.3758/BF03212325
Crossref   Google Scholar

Erdin, H. O. (2021). Appraisal of certain methodologies in cognitive science based on Lakatos’s methodology of scientific research programmes. Synthese, 199, 89–112. https://doi.org/ 10.1007/s11229-020-02612-4
Crossref   Google Scholar

Eronen, M. I., & Bringmann, L. F. (2021). The Theory Crisis in Psychology: How to Move Forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/ 10.1177/1745691620970586
Crossref   Google Scholar

Ferguson, C. J., & Heene, M. (2012). A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null. Perspectives on Psychological Science, 7(6), 555–561. https://doi.org/ 10.1177/1745691612459059
Crossref   Google Scholar

Fiedler, K. (1991). Heuristics and Biases in Theory Formation: On the Cognitive Processes of those Concerned with Cognitive Processes. Theory & Psychology, 1(4), 407–430. https://doi.org/ 10.1177/0959354391014002
Crossref   Google Scholar

Fiedler, K. (2017). What Constitutes Strong Psychological Science? The (Neglected) Role of Diagnosticity and A Priori Theorizing: Perspectives on Psychological Science, 12(1), 46–61. https://doi.org/ 10.1177/1745691616654458
Crossref   Google Scholar

Flis, I. (2019). Psychologists psychologizing scientific psychology: An epistemological reading of the replication crisis. Theory & Psychology, 29(2), 158–181. https://doi.org/ 10.1177/0959354319835322
Crossref   Google Scholar

Frankenhuis, W. E., Panchanathan, K., & Smaldino, P. E. (2022). Strategic ambiguity in the social sciences. Social Psychological Bulletin. https://www.psycharchives.org/en/item/e5bb9192-80a4-4ae4-9cda-5d144008196e
Crossref   Google Scholar

Fried, E. I. (2020). Lack of Theory Building and Testing Impedes Progress in The Factor and Network Literature. Psychological Inquiry, 31(4), 271–288. https://doi.org/ 10.1080/1047840X.2020.1853461
Crossref   Google Scholar

Frixione, M. (2001). Tractable competence. Minds and Machines, 11, 379–397.
Crossref   Google Scholar

Gaj, N. (2016). Unity and Fragmentation in Psychology: The Philosophical and Methodological Roots of the Discipline. Taylor & Francis Ltd.
Crossref   Google Scholar

Giere, R. N., & Moffatt, B. (2003). Distributed Cognition: Where the Cognitive and the Social Merge. Social Studies of Science, 33(2), 301–310. https://doi.org/ 10.1177/03063127030332017
Crossref   Google Scholar

Gigerenzer, G. (1991). From tools to theories: A heuristic of discovery in cognitive psychology. Psychological Review, 98(2), 254–267. https://doi.org/ 10.1037/0033-295X.98.2.254
Crossref   Google Scholar

Gigerenzer, G. (1992). Discovery in Cognitive Psychology: New Tools Inspire New Theories. Science in Context, 5(2), 329–350. https://doi.org/ 10.1017/S0269889700001216
Crossref   Google Scholar

Gigerenzer, G. (1998). Surrogates for Theories. Theory & Psychology, 8(2), 195–204. https://doi.org/ 10.1177/0959354398082006
Crossref   Google Scholar

Gitelman, L. (Ed.). (2013). “Raw data” is an oxymoron. The MIT Press.
Crossref   Google Scholar

Goertzen, J. R. (2008). On the Possibility of Unification: The Reality and Nature of the Crisis in Psychology. Theory & Psychology, 18(6), 829–852. https://doi.org/ 10.1177/0959354308097260
Crossref   Google Scholar

Gorelick, R. (2011). What is theory? Ideas in Ecology and Evolution, 4, 1–10. https://doi.org/ 10.4033/iee.2011.4.1.c
Crossref   Google Scholar

Greenwald, A. G. (2012). There Is Nothing So Theoretical as a Good Method: Perspectives on Psychological Science, 7(2), 99–108. https://doi.org/ 10.1177/1745691611434210
Crossref   Google Scholar

Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress? Psychological Review, 93(2), 216–229. https://doi.org/ 10.1037/0033-295X.93.2.216
Crossref   Google Scholar

Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., … Zwienenberg, M. (2016). A Multilab Preregistered Replication of the Ego-Depletion Effect. Perspectives on Psychological Science, 11(4), 546–573. https://doi.org/ 10.1177/1745691616652873
Crossref   Google Scholar

Hensel, W. M. (2020). Double trouble? The communication dimension of the reproducibility crisis in experimental psychology and neuroscience. European Journal for Philosophy of Science, 10(3), 44. https://doi.org/ 10.1007/s13194-020-00317-6
Crossref   Google Scholar

Hensel, W. M., Miłkowski, M., & Nowakowski, P. (2022). Without more theory, psychology will be a headless rider. Behavioral and Brain Sciences, 45, e20. https://doi.org/ 10.1017/S0140525X21000212
Crossref   Google Scholar

Hoyningen-Huene, P. (2013). Systematicity: The Nature of Science. Oxford University Press.
Crossref   Google Scholar

Hughes, B. M. (2018). Psychology in crisis. Palgrave.   Google Scholar

Hutchins, E. (1995). Cognition in the wild. MIT Press.
Crossref   Google Scholar

Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/ 10.1371/journal.pmed.0020124
Crossref   Google Scholar

Irvine, E. (2021). The Role of Replication Studies in Theory Building. Perspectives on Psychological Science, 16(4), 844–853. https://doi.org/ 10.1177/1745691620970558
Crossref   Google Scholar

Isaac, A. M. C. (2019). Epistemic Loops and Measurement Realism. Philosophy of Science, 86(5), 930–941. https://doi.org/ 10.1086/705476
Crossref   Google Scholar

Ivani, S. (2019). What we (should) talk about when we talk about fruitfulness. European Journal for Philosophy of Science, 9(4), 1–18. https://doi.org/ 10.1007/s13194-018-0231-7
Crossref   Google Scholar

Kawa, S., & Giordano, J. (2012). A brief historicity of the Diagnostic and Statistical Manual of Mental Disorders: Issues and implications for the future of psychiatric canon and practice. Philosophy, Ethics, and Humanities in Medicine: PEHM, 7, 2. https://doi.org/ 10.1186/1747-5341-7-2
Crossref   Google Scholar

Keas, M. N. (2018). Systematizing the theoretical virtues. Synthese, 195(6), 2761–2793. https://doi.org/ 10.1007/s11229-017-1355-6
Crossref   Google Scholar

Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., … Suchomel, V. (2014). The Sketch Engine: Ten years on. Lexicography, 1, 7–36. https://doi.org/ 10.1007/s40607-014-0009-9
Crossref   Google Scholar

Klein, S. B. (2014). What can recent replication failures tell us about the theoretical commitments of psychology? Theory & Psychology, 24(3), 326–338. https://doi.org/ 10.1177/0959354314529616
Crossref   Google Scholar

Koyré, A. (1953). An Experiment in Measurement. Proceedings of the American Philosophical Society, 97(2), 222–237.   Google Scholar

Kuhn, T. S. (1977). The essential tension: Selected studies in scientific tradition and change. The University of Chicago Press.
Crossref   Google Scholar

Langley, P., Simon, H. A., Bradshaw, G. L., & Żytkow, J. M. (1987). Scientific discovery: Computational explorations of the creative processes. MIT Press.
Crossref   Google Scholar

Laudan, L. (1984). Science and values: The aims of science and their role in scientific debate. University of California Press.   Google Scholar

Lean, O. M., Rivelli, L., & Pence, C. H. (2021). Digital Literature Analysis for Empirical Philosophy of Science. The British Journal for the Philosophy of Science. https://doi.org/ 10.1086/715049   Google Scholar

Leonelli, S. (2016). Data-centric biology: A philosophical study. The University of Chicago Press.
Crossref   Google Scholar

Levenstein, D., Alvarez, V. A., Amarasingham, A., Azab, H., Chen, Z. S., Gerkin, R. C., … Redish, A. D. (2023). On the Role of Theory and Modeling in Neuroscience. Journal of Neuroscience, 43(7), 1074–1088. https://doi.org/ 10.1523/JNEUROSCI.1179-22.2022
Crossref   Google Scholar

Litwin, P., & Miłkowski, M. (2020). Unification by Fiat: Arrested Development of Predictive Processing. Cognitive Science, 44(7), e12867. https://doi.org/ 10.1111/cogs.12867
Crossref   Google Scholar

Longino, H. E. (1996). Cognitive and Non-Cognitive Values in Science: Rethinking the Dichotomy. In L. H. Nelson & J. Nelson (Eds.), Feminism, Science, and the Philosophy of Science (pp. 39–58). Springer Netherlands. https://doi.org/ 10.1007/978-94-009-1742-2_3
Crossref   Google Scholar

MacCorquodale, K. (1970). On Chomsky’s review of Skinner’s Verbal Behavior. Journal of the Experimental Analysis of Behavior, 13(1), 83–99. https://doi.org/ 10.1901/jeab.1970.13-83
Crossref   Google Scholar

Manninen, T., Aćimović, J., Havela, R., Teppola, H., & Linne, M.-L. (2018). Challenges in Reproducibility, Replicability, and Comparability of Computational Models and Tools for Neuronal and Glial Networks, Cells, and Subcellular Structures. Frontiers in Neuroinformatics, 12, 20. https://doi.org/ 10.3389/fninf.2018.00020
Crossref   Google Scholar

Marr, D. (1982). Vision. W. H. Freeman and Company.   Google Scholar

Matthews, G. (2020). Against consensus: Embracing the disunity of personality theory. Personality and Individual Differences, 152, 109535. https://doi.org/ 10.1016/j.paid.2019.109535
Crossref   Google Scholar

McMullin, E. (2010). The Virtues of a Good Theory. In The Routledge Companion to Philosophy of Science. Routledge. https://doi.org/ 10.4324/9780203744857.ch53
Crossref   Google Scholar

Meehl, P. E. (1967). Theory-Testing in Psychology and Physics: A Methodological Paradox. Philosophy of Science, 34(2), 103–115. https://doi.org/ 10.1086/288135
Crossref   Google Scholar

Miłkowski, M. (2019). Fallible Heuristics and Evaluation of Research Traditions. The Case of Embodied Cognition. Ruch Filozoficzny, 75(2), 223–236. https://doi.org/ 10.12775/RF.2019.031
Crossref   Google Scholar

Miłkowski, M. (2022). Cognitive Artifacts and Their Virtues in Scientific Practice. Studies in Logic, Grammar and Rhetoric, 67(3), 219–246. https://doi.org/ 10.2478/slgr-2022-0012
Crossref   Google Scholar

Miłkowski, M., Hensel, W. M., & Hohol, M. (2018). Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail. Journal of Computational Neuroscience, 45(3), 163–172. https://doi.org/ 10.1007/s10827-018-0702-z
Crossref   Google Scholar

Miłkowski, M., & Litwin, P. (2022). Testable or bust: Theoretical lessons for predictive processing. Synthese, 200(6), 462. https://doi.org/ 10.1007/s11229-022-03891-9
Crossref   Google Scholar

Mischel, W. (2008). The Toothbrush Problem. APS Observer, 21(11). Retrieved from https://www.psychologicalscience.org/observer/the-toothbrush-problem   Google Scholar

Moretti, F. (2000). Conjectures on World Literature. New Left Review, 1, 54–68.   Google Scholar

Morgan, M. S., & Morrison, M. (1999). Models As Mediators. Cambridge University Press.
Crossref   Google Scholar

Muthukrishna, M., & Henrich, J. (2019). A problem in theory. Nature Human Behaviour, 3, 221–229. https://doi.org/ 10.1038/s41562-018-0522-1
Crossref   Google Scholar

Nersessian, N. J. (2008). Creating scientific concepts. MIT Press.
Crossref   Google Scholar

Newell, A. (1973). You can’t play 20 questions with nature and win: Projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing (pp. 283–308). Academic Press.
Crossref   Google Scholar

Newell, A., & Simon, H. A. (1972). Human Problem Solving. Prentice-Hall.   Google Scholar

Nickles, T. (2018). TTT: A Fast Heuristic to New Theories? In D. Danks & E. Ippoliti (Eds.), Building Theories: Heuristics and Hypotheses in Sciences (pp. 169–189). Springer. https://doi.org/ 10.1007/978-3-319-72787-5_9
Crossref   Google Scholar

Norman, D. A. (1991). Cognitive Artifacts. In J. M. Carroll (Ed.), Designing Interaction: Psychology at the Human-Computer Interface (pp. 17–38). Cambridge University Press.   Google Scholar

Norton, J. D. (2021). The Material Theory of Induction. University of Calgary Press.
Crossref   Google Scholar

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., … Vazire, S. (2022). Replicability, Robustness, and Reproducibility in Psychological Science. Annual Review of Psychology, 73(1), 719–748. https://doi.org/ 10.1146/annurev-psych-020821-114157
Crossref   Google Scholar

Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/ 10.3758/s13423-019-01645-2
Crossref   Google Scholar

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251). https://doi.org/ 10.1126/science.aac4716
Crossref   Google Scholar

Osbeck, L. M., & Nersessian, N. J. (2014). Situating distributed cognition. Philosophical Psychology, 27(1), 82–97. https://doi.org/ 10.1080/09515089.2013.829384
Crossref   Google Scholar

Pence, C. H., & Ramsey, G. (2018). How to Do Digital Philosophy of Science. Philosophy of Science, 85(5), 930–941. https://doi.org/ 10.1086/699697
Crossref   Google Scholar

Piper, A. (2020). Can We Be Wrong? The Problem of Textual Evidence in a Time of Data. Cambridge University Press.
Crossref   Google Scholar

Poldrack, R. A., Kittur, A., Kalar, D., Miller, E., Seppa, C., Gil, Y., … Bilder, R. M. (2011). The Cognitive Atlas: Toward a Knowledge Foundation for Cognitive Neuroscience. Frontiers in Neuroinformatics, 5. https://doi.org/ 10.3389/fninf.2011.00017
Crossref   Google Scholar

Rescher, N. (1979). Cognitive systematization: A systems-theoretic approach to a coherentist theory of knowledge. Basil Blackwell.   Google Scholar

Roberts, S., & Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107(2), 358–358.
Crossref   Google Scholar

Rosenhan, D. L. (1973). On Being Sane in Insane Places. Science, 179(4070), 250–258. https://doi.org/ 10.1126/science.179.4070.250
Crossref   Google Scholar

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/ 10.1037/0033-2909.86.3.638
Crossref   Google Scholar

Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2020). Why Hypothesis Testers Should Spend Less Time Testing Hypotheses. Perspectives on Psychological Science, 16(4), 744–755. https://doi.org/ 10.1177/1745691620966795
Crossref   Google Scholar

Schindler, S. (2018). Theoretical virtues in science: Uncovering reality through theory. Cambridge University Press.
Crossref   Google Scholar

Schooler, J. W. (2014). Metascience could rescue the ‘replication crisis.’ Nature, 515(7525), 9. https://doi.org/ 10.1038/515009a
Crossref   Google Scholar

Scull, A. (2023). Rosenhan revisited: Successful scientific fraud. History of Psychiatry, 0957154X221150878. https://doi.org/ 10.1177/0957154X221150878
Crossref   Google Scholar

Shmueli, G. (2010). To Explain or to Predict? Statistical Science, 25(3), 289–310. https://doi.org/ 10.1214/10-STS330
Crossref   Google Scholar

Shmueli, G., & Koppius, O. R. (2011). Predictive Analytics in Information Systems Research. MIS Quarterly, 35(3), 553. https://doi.org/ 10.2307/23042796
Crossref   Google Scholar

Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., … Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/ 10.1177/2515245917747646
Crossref   Google Scholar

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/ 10.1177/0956797611417632
Crossref   Google Scholar

Smaldino, P. E. (2017). Models Are Stupid, and We Need More of Them. In R. R. Vallacher, S. J. Read, & A. Nowak (Eds.), Computational Social Psychology (1st ed., pp. 311–331). Routledge. https://doi.org/ 10.4324/9781315173726-14
Crossref   Google Scholar

Staats, A. W. (1986). Unified Positivism: A Philosophy for Psychology and the Disunified Sciences. Theoretical & Philosophical Psychology, 6(2), 77–90. https://doi.org/ 10.1037/h0091427
Crossref   Google Scholar

Sullivan, J. A. (2009). The multiplicity of experimental protocols: A challenge to reductionist and non-reductionist models of the unity of neuroscience. Synthese, 167(3), 511–539. https://doi.org/ 10.1007/s11229-008-9389-4
Crossref   Google Scholar

Suppes, P. (1962). Models of Data. In E. Nagel, P. Suppes, & A. Tarski (Eds.), Logic, Methodology, and Philosophy of Science: Proceedings of the 1960 International Congress (pp. 252–261). Stanford University Press.
Crossref   Google Scholar

Szollosi, A., & Donkin, C. (2019). Neglected Sources of Flexibility in Psychological Theories: From Replicability to Good Explanations. Computational Brain & Behavior, 2(3–4), 190–192. https://doi.org/ 10.1007/s42113-019-00045-y
Crossref   Google Scholar

Thagard, P. (1993). Computational philosophy of science. MIT Press.   Google Scholar

Thagard, P., & Findlay, S. (2012). The cognitive science of science: Explanation, discovery, and conceptual change. MIT Press.
Crossref   Google Scholar

Trafimow, D., & Earp, B. D. (2016). Badly specified theories are not responsible for the replication crisis in social psychology: Comment on Klein. Theory & Psychology, 26(4), 540–548. https://doi.org/ 10.1177/0959354316637136
Crossref   Google Scholar

Vadillo, M. A. (2019). Ego depletion may disappear by 2020. Social Psychology, 50, 282–291. https://doi.org/ 10.1027/1864-9335/a000375
Crossref   Google Scholar

Vadillo, M. A., Gold, N., & Osman, M. (2016). The Bitter Truth About Sugar and Willpower: The Limited Evidential Value of the Glucose Model of Ego Depletion. Psychological Science, 27(9), 1207–1214. https://doi.org/ 10.1177/0956797616654911
Crossref   Google Scholar

Van Rooij, I. (2008). The Tractable Cognition Thesis. Cognitive Science, 32(6), 939–984. https://doi.org/ 10.1080/03640210801897856
Crossref   Google Scholar

Van Rooij, I., & Baggio, G. (2021). Theory Before the Test: How to Build High-Verisimilitude Explanatory Theories in Psychological Science. Perspectives on Psychological Science, 16(4), 682–697. https://doi.org/ 10.1177/1745691620970604
Crossref   Google Scholar

Vohs, K. D., Schmeichel, B. J., Lohmann, S., Gronau, Q. F., Finley, A. J., Ainsworth, S. E., … Albarracín, D. (2021). A Multisite Preregistered Paradigmatic Test of the Ego-Depletion Effect. Psychological Science. (Sage CA: Los Angeles, CA). https://doi.org/ 10.1177/0956797621989733
Crossref   Google Scholar

Wilson, M. (1993). DSM-III and the transformation of American psychiatry: A history. The American Journal of Psychiatry, 150(3), 399–410. https://doi.org/ 10.1176/ajp.150.3.399
Crossref   Google Scholar

Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, e1. https://doi.org/ 10.1017/S0140525X20001685
Crossref   Google Scholar

Yarkoni, T., & Westfall, J. (2017). Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/ 10.1177/1745691617693393
Crossref   Google Scholar

Young, G. (2016). Unifying Causality and Psychology. Springer International Publishing. https://doi.org/ 10.1007/978-3-319-24094-7
Crossref   Google Scholar

Zhang, J., & Norman, D. A. (1994). Representations in Distributed Cognitive Tasks. Cognitive Science, 18(1), 87–122. https://doi.org/ 10.1207/s15516709cog1801_3
Crossref   Google Scholar

Zittoun, T., Gillespie, A., & Cornish, F. (2009). Fragmentation or Differentiation: Questioning the Crisis in Psychology. Integrative Psychological and Behavioral Science, 43(2), 104–115. https://doi.org/ 10.1007/s12124-008-9083-6
Crossref   Google Scholar


Opublikowane
2023-10-26

Cited By /
Share

Miłkowski, M. (2023). Cognitive Metascience: A New Approach to the Study of Theories. Przegląd Psychologiczny, 66(1), 185–207. https://doi.org/10.31648/przegldpsychologiczny.9682

Marcin Miłkowski 
Polish Academy of Sciences, Institute of Philosophy and Sociology
https://orcid.org/0000-0001-7646-5742