Samples in Psychological Research – Conclusions and Recommendations from the Literature Review

Joanna Świderska

The Maria Grzegorzewska University, Institute of Psychology
https://orcid.org/0000-0001-7439-2973

Magdalena Puchalska

The Maria Grzegorzewska University, Institute of Psychology
https://orcid.org/0000-0001-6116-4656

Maciej Grzegorczyk

Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0004-6325-7767

Zuzanna Szałek

Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0008-0699-1374

Natalia Nadolna

Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0009-1047-4377

Martyna Andryszkiewicz

University of Lodz, Institute of Psychology
https://orcid.org/0009-0008-7433-6013

Maria Jastrzębska

Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0005-6479-7287

Przemysław Szablowski


https://orcid.org/0009-0008-7728-9339

Maja Śmigrodzka

The Maria Grzegorzewska University, Institute of Psychology
https://orcid.org/0009-0005-9517-5253


Abstract

Objective: The goal of the review was to identify the elements occurring in the both survey procedure and sample descriptions and to formulate recommendations for the authors on this basis.

Method: The study was conducted following the PRISMA standard. A total of 305 articles by Polish researchers from four journals were analyzed. Critical evaluation have been conducted for 230 articles of the analyzed sample, encompassing 295 research descriptions.

Results: Analysis revealed that researchers most frequently report the research procedure, sampling method, sample size, form of the research and sociodemographic characteristics, such as: gender or sex and age.

Conclusions: The conclusions drawn from the analysis and insights gained from the review led to the formulating recommendations regarding comprehensive methodological description of research procedures and samples.


Keywords:

sampling, research procedure description, data quality, generalization, literature review


Adam, A. M. (2020). Sample Size Determination in Survey Research. Journal of Scientific Research and Reports, 26(5), 90–97. https://doi.org/10.9734/jsrr/2020/v26i530263
Crossref   Google Scholar

Babbie, E. (2021). Badania społeczne w praktyce [The Practice of Social Research]. Wydawnictwo Naukowe PWN.   Google Scholar

Baker, R., Brick, J. M., Bates, N. A., Battaglia, M., Couper, M. P., Dever, J. A., Gile, K. J., & Tourangeau, R. (2013). Summary Report of the AAPOR Task Force on Non-probability Sampling. Journal of Survey Statistics and Methodology, 1(2), 90–143. https://doi.org/10.1093/jssam/smt008
Crossref   Google Scholar

Bayat, D., Mohamadpour, H., Fang, H., Xu, P., & Krueger, F. (2023). The Impact of Order Effects on the Framing of Trust and Reciprocity Behaviors. Games, 14(2), Article 21. https://doi.org/10.3390/g14020021
Crossref   Google Scholar

Bethlehem, J. (2010). Selection bias in web surveys. International Statistical Review, 78(2), 161–188. https://doi.org/10.1111/j.1751-5823.2010.00112.x
Crossref   Google Scholar

Brase, G. L. (2009). How different types of participant payments alter task performance. Judgment and Decision Making, 4(5), 419–428. https://doi.org/10.1017/S1930297500001248
Crossref   Google Scholar

Brzeziński, J. (2019). Metodologia badań psychologicznych. Wydanie nowe [Methodology of the Psychological Research. New Edition]. Wydawnictwo Naukowe PWN.   Google Scholar

Cheung, K. L., ten Klooster, P. M., Smit, C., de Vries, H., & Pieterse, M. E. (2017). The impact of non-response bias due to sampling in public health studies: A comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health, 17, Article 276. https://doi.org/10.1186/s12889-017-4189-8
Crossref   Google Scholar

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
Crossref   Google Scholar

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152X
Crossref   Google Scholar

Kaźmierczak, I., Zajenkowska, A., Rogoza, R., Jonason, P. K., & Ścigała, D. (2023). Self-selection biases in psychological studies: Personality and affective disorders are prevalent among participants. PLOS ONE, 18(3), Article e0281046. https://doi.org/10.1371/journal.pone.0281046
Crossref   Google Scholar

Komunikat Ministra Edukacji i Nauki z dnia 17 lipca 2023 r. w sprawie wykazu czasopism naukowych i recenzowanych materiałów z konferencji międzynarodowych [Polish The Communiqué the Minister of Education and Science on the list of scientific journals and peer-reviewed materials from international conferences of 17 July 2023]. (2023). https://www.gov.pl/web/edukacja-i-nauka/komunikat-ministra-edukacji-i-nauki-z-dnia-17-lipca-2023-r-w-sprawie-wykazu-czasopism-naukowych-i-recenzowanych-materialow-z-konferencji-miedzynarodowych   Google Scholar

Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1), Article 33267. https://doi.org/10.1525/collabra.33267
Crossref   Google Scholar

Lusinchi, D. (2018). ‘The Great Fiasco’ of the 1948 presidential election polls: status recognition and norms conflict in social science. Annals of Science, 75(2), 120–144. https://doi.org/10.1080/00033790.2018.1466194
Crossref   Google Scholar

Maslow, A. H., & Sakoda, J. M. (1952). Volunteer-error in the Kinsey study. The Journal of Abnormal and Social Psychology, 47(2), 259–262. https://doi.org/10.1037/h0054411
Crossref   Google Scholar

Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLOS Medicine, 6(7), Article e1000097. https://doi.org/10.1371/journal.pmed.1000097
Crossref   Google Scholar

Nikel, Ł. (2021). Polska adaptacja Skali do badania motywacji uczniów w szkole podstawowej [Polish Adaptation of the Elementary School Motivation Scale]. Polskie Forum Psychologiczne [Polish Psychological Forum], 26(1), 97–112. https://doi.org/10.34767/PFP.2021.01.06   Google Scholar

Salganik, M. J., & Heckathorn, D. D. (2004). Sampling and Estimation in Hidden Populations Using Respondent-Driven Sampling. Sociological Methodology, 34(1), 193–240. https://doi.org/10.1111/j.0081-1750.2004.00152.x
Crossref   Google Scholar

Sapra, R. L. (2022). How to Calculate an Adequate Sample Size? In S. Nundy, A. Kakar, & Z. A. Bhutta (Eds.), How to Practice Academic Medicine and Publish from Developing Countries? A Practical Guide (pp. 81–93). Springer Singapore. https://doi.org/10.1007/978-981-16-5248-6
Crossref   Google Scholar

Sarstedt, M., Bengart, P., Shaltoni, A. M., & Lehmann, S. (2018). The use of sampling methods in advertising research: a gap between theory and practice. International Journal of Advertising, 37(4), 650–663. https://doi.org/10.1080/02650487.2017.1348329
Crossref   Google Scholar

Singh, A. S., & Masuku, M. B. (2014). Sampling techniques and determination of sample size in applied statistics research: An overview. International Journal of Economics, Commerce and Management, 2(11), 1–22.   Google Scholar

Shaughnessy, J. J., Zechmeister, E. B., & Zechmeister, J. S. (2002). Metody badawcze w psychologii [Research methods in psychology]. Gdańskie Wydawnictwo Psychologiczne.   Google Scholar

Squire, P. (1988). Why the 1936 Literary Digest Poll Failed. The Public Opinion Quarterly, 52(1), 125–133. http://www.jstor.org/stable/2749114
Crossref   Google Scholar

Thompson, V. A., & Campbell, J. I. D. (2004). A power struggle: Between- vs. within-subjects designs in deductive reasoning research. Psychologia: An International Journal of Psychology in the Orient, 47(4), 277–296. https://doi.org/10.2117/psysoc.2004.277
Crossref   Google Scholar

Ustawa z dnia 19 listopada 2009 r. o grach hazardowych Dz.U. 2009 nr 201, poz. 1540 [Polish The Act on Gambling Games of 19 November 2009, Journal of Laws of 2009, No. 201, item 1540]. (2019). https://isap.sejm.gov.pl/isap.nsf/DocDetails.xsp?id=WDU20092011540   Google Scholar

Wojciszke, B. (2004). Systematycznie modyfikowane autoreplikacje: logika programu badań empirycznych w psychologii [Systematically Modified Auto-Replications: the logic of the empirical research program in psychology]. In J. M. Brzeziński (Ed.), Metodologia badań psychologicznych. Wybór tekstów [Methodology of the Psychological Research. Selected papers] (pp. 44–68). Wydawnictwo Zysk i Ska.   Google Scholar

Zimbardo, P. G., & Gerrig, R. J. (2012). Psychologia i życie [Psychology and Life]. Wydawnictwo Naukowe PWN.   Google Scholar

Download


Published
2023-12-22

Cited by

Świderska, J., Puchalska, M., Grzegorczyk, M., Szałek, Z., Nadolna, N., Andryszkiewicz, M., Jastrzębska, M., Szablowski, P., & Śmigrodzka, M. (2023). Samples in Psychological Research – Conclusions and Recommendations from the Literature Review. The Review of Psychology, 66(3), 111–132. https://doi.org/10.31648/przegldpsychologiczny.9831

Joanna Świderska 
The Maria Grzegorzewska University, Institute of Psychology
https://orcid.org/0000-0001-7439-2973
Magdalena Puchalska 
The Maria Grzegorzewska University, Institute of Psychology
https://orcid.org/0000-0001-6116-4656
Maciej Grzegorczyk 
Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0004-6325-7767
Zuzanna Szałek 
Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0008-0699-1374
Natalia Nadolna 
Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0009-1047-4377
Martyna Andryszkiewicz 
University of Lodz, Institute of Psychology
https://orcid.org/0009-0008-7433-6013
Maria Jastrzębska 
Cardinal Stefan Wyszyński University in Warsaw, Institute of Psychology
https://orcid.org/0009-0005-6479-7287
Maja Śmigrodzka 
The Maria Grzegorzewska University, Institute of Psychology
https://orcid.org/0009-0005-9517-5253