When Cheating is an Honest Mistake: A Critical Evaluation of the Matrix Task as a Measure of Dishonesty

Abstract

Dishonesty is an intriguing phenomenon, studied extensively across various disciplines due to its impact on people's lives as well as society in general. To examine dishonesty in a controlled setting, researchers have developed a number of experimental paradigms. One of the most popular approaches in this regard, is the matrix task, in which participants receive matrices wherein they have to find two numbers that sum to 10 (e.g., 4.81 and 5.19), under time pressure. In a next phase, participants need to report how many matrices they had solved correctly, allowing them the opportunity to cheat by exaggerating their performance in order to get a larger reward. Here, we argue, both on theoretical and empirical grounds, that the matrix task is ill-suited to study dishonest behavior, primarily because it conflates cheating with honest mistakes. We therefore recommend researchers to use different paradigms to examine dishonesty, and treat (previous) findings based on the matrix task with due caution.

Dark, gray, or bright creativity? (Re)investigating the link between creativity and dishonesty

“…Two of the major strengths of this study are the large sample size accumulated across labs and Arguably even more importantly, the matrix task does not assess pure dishonesty because the final score not only represents the number of matrices that were dishonestly reported as solved but also the number of matrices that were actually solved correctly. Furthermore, some participants might have unintentionally thought to have solved a specific matrix (by miscalculating, maybe due to the time pressure; Heyman et al, 2020). This limitation might explain, for instance, the very small relation between Honesty-Humility and cheating, given that previous studies consistently found larger effects Zettler et al, 2020).…”

Honesty-Humility Negatively Correlates with Dishonesty in Romantic Relationships

“…Even we increased the ecological validity in Study 8 compared to the studies before, the used method of Study 8 still relied on participants self-report to assess relationship-based dishonesty. In the past, deception research revealed further paradigms that base the dishonesty assessment on participants' active decision to behave deceptively or non-deceptively in concrete situations (e.g., Daumiller & Janke, 2020;Heyman et al, 2020). Accordingly, we also tested a number of different paradigms in which we measured actual relationship-based dishonest behavior in concrete situations (see Deception Paradigms in the OSF); we applied the most suitable deception paradigm in Study 9.…”

“…Although anagram tasks are widely used and established in deception research (Gerlach et al, 2019), Heyman et al (2020) asserted that dishonesty could be conflated with honest mistakes, meaning that participants are unaware of their overreporting because they mistakenly thought that they found a correct solution.…”

“…However, considering this problem, and according to the recommendations of Heyman et al (2020), Study 11 applied a die roll task as a final test of our hypotheses.…”

Do early birds share their worms? How empathy and prosociality vary across the course of the day

“…Social behaviours in everyday life rarely resemble the economic games or lab-based tasks used to measure morality and prosociality in the lab. Rates of dishonesty in lab-based tasks do not resemble rates of dishonesty in comparable field studies (Gerlach et al, 2019), and the matrix task, often used by the above studies as a measure of dishonesty, may also be capturing honest mistakes in addition to intentional lies (Heyman et al, 2020). Recent studies have thus sought more ecologically valid evidence for circadian effects on prosocial behaviour, with surprising results.…”