A screen-time-based mixture model for identifying and monitoring careless and insufficient effort responding in ecological momentary assessment data.

Ulitzsch, Esther; Nestler, Steffen; Lüdtke, Oliver; Nagy, Gabriel

doi:10.1037/met0000636

Cited by 4 publications

(10 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This overcomes the need for post-hoc interpretation of possibly ambiguous latent states resulting from the exploratory models used in regular LMFA. As previously developed mixture modeling approaches to C/IER in EMA data (Hasselhorn et al, 2023;Ulitzsch et al, 2024), this new method does not require decisions on threshold settings and takes C/IER classification uncertainty into account. The method is applicable to EMA studies employing multiple items with ordinal response scales to measure psychological constructs.…”

Section: Proposed Method: Lmfa With Confirmatory Mixture Irt Modelsmentioning

confidence: 99%

“…In this approach, classes with aberrant conditional means on the analyzed indicators are labeled as C/IER. Note, however, that post-hoc interpretation of the obtained latent classes may not always be straightforward (see Ulitzsch et al, 2024, for further examples and discussions). Hence, despite these recent advances, a common limitation of approaches relying on behavioral indicators remains that they are only loosely grounded in subject-matter theory on respondent behavior.…”

Section: Indicators Based On Response Patterns and Screen Timesmentioning

confidence: 99%

“…Only recently, Ulitzsch et al (2024) provided a confirmatory mixture modeling approach for C/IER tailored to the EMA context. This model is formulated for screen times instead of for item responses.…”

Section: Confirmatory Mixture Model Based On Screen Timesmentioning

confidence: 99%

“…In the present study, we focus on careless and insufficient effort responding (C/IER) as a prominent form of respondent non-compliance. C/IER occurs when respondents complete questionnaires without investing effort into carefully evaluating the administered items (Huang et al, 2015;Ulitzsch et al, 2024). As a consequence, EMA data may be contaminated with responses that do not reflect what researchers want to measure.…”

mentioning

confidence: 99%

“…to the EMA context emerged only recently and have, so far, only been developed for collateral information from screen time data (Ulitzsch et al, 2024).…”

mentioning

confidence: 99%

See 4 more Smart Citations

Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data

Vogelsmeier,

Uglanova,

Rein

et al. 2024

Preprint

View full text Add to dashboard Cite

In ecological momentary assessment (EMA), respondents answer brief questionnaires about their current behaviors or experiences several times per day across multiple days. The frequent measurement enables a thorough grasp of the dynamics inherent in psychological traits, but it also increases respondent burden. To lower this burden, respondents may engage in careless and insufficient effort responding (C/IER) and leave data contaminated with responses that do not reflect what researchers want to measure. We introduce a novel approach to investigate C/IER in EMA data. Our approach combines a confirmatory mixture item response theory model separating C/IER from attentive behavior with latent Markov factor analysis. This allows for (1) gauging the occurrence of C/IER and (2) studying transitions among states of different response behaviors as well as their contextual correlates. The approach can be implemented using standard R packages. In an empirical application, we showcase the efficacy of this approach in both pinpointing C/IER instances in EMA and gaining insights into their underlying causes. In a simulation study investigating robustness against unaccounted changes in measurement models underlying attentive responses, the approach proved robust against heterogeneity in loading patterns but not against heterogeneity in the factor structure. Extensions to accommodate the latter are discussed.

show abstract

Section: Proposed Method: Lmfa With Confirmatory Mixture Irt Modelsmentioning

confidence: 99%

Section: Indicators Based On Response Patterns and Screen Timesmentioning

confidence: 99%

Section: Confirmatory Mixture Model Based On Screen Timesmentioning

confidence: 99%

mentioning

confidence: 99%

“…to the EMA context emerged only recently and have, so far, only been developed for collateral information from screen time data (Ulitzsch et al, 2024).…”

mentioning

confidence: 99%

See 3 more Smart Citations

Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data

Vogelsmeier,

Uglanova,

Rein

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data

Vogelsmeier,

Uglanova,

Rein

et al. 2024

Brit J Math & Statis

View full text Add to dashboard Cite

In ecological momentary assessment (EMA), respondents answer brief questionnaires about their current behaviours or experiences several times per day across multiple days. The frequent measurement enables a thorough grasp of the dynamics inherent in psychological constructs, but it also increases respondent burden. To lower this burden, respondents may engage in careless and insufficient effort responding (C/IER), leaving data contaminated with responses that do not reflect what researchers want to measure. We introduce a novel approach to investigating C/IER in EMA data. Our approach combines a confirmatory mixture item response theory model separating C/IER from attentive behaviour with latent Markov factor analysis. This enables gauging the occurrence of C/IER and studying transitions among states of different response behaviours including their contextual correlates. The approach can be implemented using R packages. An empirical application showcases the approach's efficacy in pinpointing C/IER instances and gaining insights into their underlying causes. We showcase that the approach identifies various C/IER response patterns but requires heterogeneous and negatively worded items to detect straightlining. In a simulation investigating robustness against unaccounted for changes in measurement models underlying attentive responses, the approach proved robust against heterogeneity in loading patterns but not against heterogeneity in factor structures. Extensions to accommodate the latter are discussed.

show abstract

Intensive, Repeated Self-Report Measures:Should We Be Concerned about Changes in Data Quality Over Time? (Preprint)

Stone,

Schneider,

Toledo

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND There are many concerns about repeated measurement with self-report methods that are discussed in this paper. These points deserve to be considered by those using intensive, repeated self-report designs. OBJECTIVE To outline the potential issues with repeated measures intensive measurement studies. METHODS This is primarily a discussion/opinion paper. RESULTS Many concerns about intensive, repeated measurement designs are discussed. CONCLUSIONS There are many concerns that researchers using this methodology should consider and, if possible, address.

show abstract

A screen-time-based mixture model for identifying and monitoring careless and insufficient effort responding in ecological momentary assessment data.

Cited by 4 publications

References 87 publications

Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data

Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data

Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data

Intensive, Repeated Self-Report Measures:Should We Be Concerned about Changes in Data Quality Over Time? (Preprint)

Contact Info

Product

Resources

About