Web-Based Research in Psychology

Abstract

Abstract. The present article reviews web-based research in psychology. It captures principles, learnings, and trends in several types of web-based research that show similar developments related to web technology and its major shifts (e.g., appearance of search engines, browser wars, deep web, commercialization, web services, HTML5…) as well as distinct challenges. The types of web-based research discussed are web surveys and questionnaire research, web-based tests, web experiments, Mobile Experience Sampling, and non-reactive web research, including big data. A number of web-based methods are presented and discussed that turned out to become important in research methodology. These are one-item-one-screen design, seriousness check, instruction manipulation and other attention checks, multiple site entry technique, subsampling technique, warm-up technique, and web-based measurement. Pitfalls and best practices are described then, especially regarding dropout and other non-response, recruitment of participants, and interaction between technology and psychological factors. The review concludes with a discussion of important concepts that have developed over 25 years and an outlook on future developments in web-based research.

Sample Size Trends in Experimental Psychology from 2003 to 2022: Soaring Online Samples and Lagging Offline Samples

“…However, while in the past years still only about one-fourth of the articles contained online samples (Figure 3), the average number of online participants we found (per article) is almost twice of that of offline participants. This result is all the more encouraging as it was found despite dropout rates being much larger in online studies than in offline studies (Reips, 2021). The different journals show similar patterns, though online samples prevail more in JEP General and Psychological Science (Figure 5).…”

“…From only a very small number of Internet-based studies until the early 2010s, during the past decade online research has soared to the extent that in the recent years more than half of the subjects in experimental psychology has participated online. The concomitantly increased total sample sizes (despite only a slight increase in offline samples) likely bode well for replicability and robustness (Nosek et al, 2022;Reips, 2000Reips, , 2021, although the precise impact would require more detailed examination (see Sassenberg & Ditrich, 2019). As widely discussed in reviews, online research has its pitfalls as well (e.g., Chandler et al, 2014;Gagné & Franzen, 2021;Reips, 2021;Zhou & Fishbach, Appendix A…”

“…However, given the sharp correspondence of the increased samples with online samples (Figure 3) and the many reasons to conduct research via the Internet (e.g. Reips, 2000Reips, , 2021, it seems likely that online experiments drive increased sample sizes.…”

jsQuestPlus: A JavaScript implementation of the QUEST+ method for estimating psychometric function parameters in online experiments

“…Santacroce et al (2021) were not planning to do their experiments online but decided to conduct part of the experiments online due to COVID-19. In addition to efficient recruitment of a large number and wide variety of participants (Reips, 2021), online experiments can proceed regardless of any lockdown measures. Considering the rise in online psychophysical experiments, there is a growing need for online experiment tools to match the functionality offered in the lab.…”

From Modems to Mobile Apps

“…In the first article, Reips (2021) reviews different types of web-based research, including research that could not be done without the web. He highlights how some specific research areas, for example, on the rare disorder sexsomnia, have profited enormously from the advantages of the web, in this case the combination of reach and anonymity.…”