Interval sampling methods and measurement error: A computer simulation

Wirth, Oliver; Slaven, James E.; Taylor, Miles

doi:10.1002/jaba.93

Cited by 32 publications

(39 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consistent with prior DC/CDC research, results from Experiment 1 indicate that the number of points required in Phase B to decrease the false positive to an acceptable level is generally between four and six points, as the occurrence of false positives is often affected by the number of data points in each data series (Devine et al, ). Results from Experiment 2 are consistent with extant research, which suggests that PIR may overestimate the occurrence of duration events, and discontinuous interval methods may yield differences in data series trends (see Wirth, Slaven, & Taylor, ). Thus, practitioners should exercise caution when using interval methods with large interval sizes.…”

Section: Discussionsupporting

confidence: 80%

Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual‐criteria methods

Falligant

Vetter

2020

Behavioral Interventions

View full text Add to dashboard Cite

The dual-criteria (DC) and conservative dual-criteria (CDC) methods allow clinicians and researchers to quantify the occurrence of false-positive outcomes within single-case experimental designs. The purpose of the current study was to use these DC and CDC methods to measure the incidence of false positives with simulated data collected via discontinuous interval methods (i.e., momentary time sampling, partialinterval recording) as a function of data series length. We generated event data to create 316,800 unique simulated data series for analysis. In Experiment 1, we evaluated how changes in relevant parameters (i.e., interval sizes, event durations, IRT-to-event-run ratios) produced false positives with momentary time sampling procedures. We also assessed the degree that the CDC method produced fewer false positives than the DC method with simulated interval data. In Experiment 2, we used similar procedures to quantify the occurrence of false positives with partial-interval recording data. We successfully replicated outcomes from previous research in the current study, though such results only highlight the generality of the procedures relating to false positive (and not false negative) outcomes. That is, these results indicate MTS and PIR may adequately control for false positives, but our conclusions are limited by a lack of data on power. K E Y W O R D S dual-criteria method, false positives, momentary time sampling, partial interval recording

show abstract

Section: Discussionsupporting

confidence: 80%

Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual‐criteria methods

Falligant

Vetter

2020

Behavioral Interventions

View full text Add to dashboard Cite

show abstract

“…A limitation of the LRR is that it remains sensitive to the use of partial interval recording systems of varying length. However, this may have less to do with the effect size measure than with the PIR system itself, which systematically over-estimates the prevalence of state behaviors, to an extent that depends both on interval length and on other features of the behavior (Kraemer, 1979;Wirth, Slaven, & Taylor, 2014). Another limitation of the LRR is that it does not account for time trends within the baseline or treatment phases.…”

Section: Discussionmentioning

confidence: 99%

Procedural sensitivities of effect sizes for single-case designs with directly observed behavioral outcome measures

Pustejovsky

2018

Preprint

View full text Add to dashboard Cite

A wide variety of effect size indices have been proposed for quantifying the magnitude of treatment effects in single-case designs. Commonly used measures include parametric indices such as the standardized mean difference, as well as non-overlap measures such as the percentage of non-overlapping data, improvement rate difference, and non-overlap of all pairs. Currently, little is known about the properties of these indices when applied to behavioral data collected by systematic direct observation, even though systematic direct observation is the most common method for outcome measurement in single-case research. This study uses Monte Carlo simulation to investigate the properties of several widely used single-case effect size measures when applied to systematic direct observation data. Results indicate that the magnitude of the non-overlap measures and of the standardized mean difference can be strongly influenced by procedural details of the study's design, which is a significant limitation to using these indices as effect sizes for meta-analysis of single-case designs. A less widely used parametric index, the log-response ratio, has the advantage of being insensitive to sample size and observation session length, although its magnitude is influenced by the use of partial interval recording.

show abstract

“…In the classification assessment, verbal skills were assessed with a randomized sequence of 30 30-s video trials, comprised of 15 trials with examples of safe leg angles and 15 trials with examples of at-risk leg angles. Consistent with the whole interval recording method (Wirth, Slaven, & Taylor, 2014), participants were instructed to classify each video as safe only if leg angles were safe for the entire video; otherwise, the videos were to be classified as at-risk. Participants recorded their classification of each video on a check sheet.…”

Section: Methodsmentioning

confidence: 99%

Experimental analysis of using examples and non-examples in safety training

Taylor

Wirth

Olvina

et al. 2016

Journal of Safety Research

Self Cite

View full text Add to dashboard Cite

Introduction The effects of training content consisting of examples and/or non-examples was studied on the acquisition of safety-related skills. Method Participants (N = 160) were randomly assigned to first receive computer-based training on office ergonomics that included either no examples of safe or at-risk postures, safe examples only, at-risk examples only, or both safe and at-risk examples. Participants then attempted to classify as safe or at-risk various postures depicted in short video clips and demonstrate with their own posture the range of safe postures. Results Groups that were trained with both safe and at-risk examples showed greater classification accuracy and less error in their demonstration of safe postures. Training with only safe or at-risk examples resulted in a moderate amount of error and a consistent underestimation of risk. Conclusion Training content consisting of both examples and non-examples improved acquisition of safety-related skills. Practical applications The strategic selection of training content may improve identification of risks and safe work practices.

show abstract

Interval sampling methods and measurement error: A computer simulation

Cited by 32 publications

References 60 publications

Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual‐criteria methods

Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual‐criteria methods

Procedural sensitivities of effect sizes for single-case designs with directly observed behavioral outcome measures

Experimental analysis of using examples and non-examples in safety training

Contact Info

Product

Resources

About