“Rarely safe to assume”: Evaluating the use and interpretation of manipulation checks in experimental social psychology

Ejelöv, Emma; Luke, Timothy J.

doi:10.1016/j.jesp.2019.103937

Cited by 87 publications

(56 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The scoping review revealed that over a third (35.67%) of studies within the review excluded participants on the basis of manipulation check, general memory check, or attention check questions included in their studies. This proportion of studies using this exclusion criterion exceeds the rates of studies employing this exclusion criterion in general social psychology research (Abbey & Meloy, 2017; Ejelöv & Luke, 2020). For some studies within the scoping review, exclusion rates were not modest, with a range from 0–34% of total participants excluded.…”

Section: Discussionmentioning

confidence: 74%

“…However, this is what is often assumed. Manipulation checks and attention checks are treated as interchangeable (Ejelöv & Luke, 2020). Our review also highlighted this, with studies using inconsistent terminology to describe memory‐ and attention‐based questions, stating that memory‐based questions were included to assess participants' attention (and excluding on this basis).…”

Section: Discussionmentioning

confidence: 99%

“…Recent reviews on manipulation check practices within social psychology research have demonstrated that studies often do not clearly define the type of question(s) asked or discriminate between different types of questions (Ejelöv & Luke, 2020; Hauser et al, 2018). For the purposes of our review, we used the following four categories to distinguish between different types of questions asked to assess memory, attention, or comprehension of mock‐jurors.…”

Section: Methodsmentioning

confidence: 99%

“…A participant who fails to answer these questions correctly may be excluded from subsequent analyses. They may be asked questions about their memory or perceptions of a particular aspect of the study that has been manipulated across participants or scenarios, commonly referred to as a manipulation check (Ejelöv & Luke, 2020). In jury research, particular details of a trial may be manipulated in order to determine the effect that detail has on decision‐making, so a manipulation check may include a participant answering a memory‐based question (or questions) about the trial.…”

Section: Introductionmentioning

confidence: 99%

“…The reasoning behind the use of questions relating to memory and attention as a basis of exclusion is one of internal validity (Ejelöv & Luke, 2020). Specifically, participants that respond incorrectly to questions relating to memory or attention may compromise the ability to draw accurate conclusions about the constructs of interest and their unique effects on jury decision‐making.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Jury simulation studies: To exclude or not to exclude participants based on a lack of comprehension of the case?

Cullen

Monds

2020

Applied Cognitive Psychology

View full text Add to dashboard Cite

Summary Jury simulation research has been the subject of longstanding criticism in regards to ecological validity. One additional factor that has received little attention that may also impact the generalizability of this research relates to excluding participants based on their memory of, or their attention paid to, the case. In order to determine how common this exclusion is, the authors conducted a scoping review of jury simulation studies within the last 10 years across three major legal psychology journals. The review revealed variability in if and how studies excluded on the basis of poor attention or for failing manipulation or general memory checks. In addition, studies that did exclude on this basis varied greatly in the proportion of the sample that was excluded. Recommendations are provided in order to assist jury researchers in future with how best to manage exclusions on the basis of juror memory and attention.

show abstract

Section: Discussionmentioning

confidence: 74%

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Jury simulation studies: To exclude or not to exclude participants based on a lack of comprehension of the case?

Cullen

Monds

2020

Applied Cognitive Psychology

View full text Add to dashboard Cite

show abstract

Stayed at home—But can't stop working despite being ill?! Guilt as a driver of presenteeism at work and home

Brosi

Gerpott

2022

J Organ Behavior

View full text Add to dashboard Cite

We argue that the literature on presenteeism needs to consider that employees not only go to work despite being ill but also often work from home despite being ill, especially since the COVID-19 crisis enabled home-office work on a large scale.We label this phenomenon "workahomeism" and develop theory that shows its distinctness from traditional presenteeism through the evoked pattern of guilt.Across three studies (a vignette experiment, a critical incident study, and a withinperson intervention study), we tested whether employees' work-related reactions to illness (i.e., workahomeism, presenteeism, and resting at home) differ in terms of experienced and anticipated guilt. We found that when employees considered engaging in workahomeism, they anticipated feeling less guilty than when resting at home. However, when employees actually engaged in workahomeism, they felt as guilty or even more guilty than when resting at home. In contrast, employees' anticipated guilt for presenteeism as compared to workahomeism changed from the same to more after the beginning of the COVID-19 crisis. Furthermore, we identify facets of guilt in response to workahomeism (i.e., guilt toward colleagues and about own health) and demonstrate that organizations can change guilt patterns by asking employees to reflect on the consequences of workahomeism and presenteeism.

show abstract

Buffer bots: The role of virtual service agents in mitigating negative effects when service fails

Sands

Campbell

Plangger

et al. 2022

Psychology and Marketing

View full text Add to dashboard Cite

In recent years, marketers have placed increased reliance upon artificial intelligence (AI) and, subsequently, the use of virtual agents in customer service contexts is on the rise. Despite such service digitalization, service can still fail. While there is an increasing literature on the effect of virtual agents in service settings, questions remain as to how customers react to service failure that results from interactions with virtual service agents. To this end, we deconstruct the effect of virtual agent service failure across two studies: one involving a process service failure and another involving an outcome service failure. We specifically manipulate the type of service agent that causes the service failure (human vs. virtual agent) and the magnitude of the failure (small vs. large). Results show that firms can leverage virtual service agents to mitigate or buffer the negative effects of service failure. From a managerial perspective, our findings suggest that firms could engage virtual service agents in situations where there may be a risk of outcome service failure-particularly in settings where relatively large magnitude failures may be experienced. In such a setting, we find that virtual service agents can mitigate the negative effects of service failure, more so than when the failure results from an interaction with a human service agent.

show abstract

“Rarely safe to assume”: Evaluating the use and interpretation of manipulation checks in experimental social psychology

Abstract: The text of the preprint may differ slightly from the version of record.

Cited by 87 publications

References 48 publications

Jury simulation studies: To exclude or not to exclude participants based on a lack of comprehension of the case?

Jury simulation studies: To exclude or not to exclude participants based on a lack of comprehension of the case?

Stayed at home—But can't stop working despite being ill?! Guilt as a driver of presenteeism at work and home

Buffer bots: The role of virtual service agents in mitigating negative effects when service fails

Contact Info

Product

Resources

About