A "Goldilocks zone" for mind wandering reports? A secondary analysis of how few thought probes are enough for reliable and valid measurement

Abstract

Mind wandering assessment relies heavily on the thought probe technique as a reliable and valid method to assess momentary task-unrelated thought (TUT), but there is little guidance available to help researchers decide how many probes to include within a task. Too few probes may lead to unreliable measurement, but too many probes might artificially disrupt normal thought flow and produce reactive effects. Is there a “Goldilocks zone” for how few thought probes can be used to reliably and validly assess individual differences in mind wandering propensity? We address this question by reanalyzing two published datasets (Study 1, n = 541; Study 2, n’s ≈ 260 per condition) in which thought probes were presented in multiple tasks. Our primary analyses randomly sampled probes in increments of two for each subject in each task. A series of confirmatory factor analyses for each probe “bin” size tested whether the latent correlations between TUT rate and theoretically relevant constructs like working memory capacity, attention control ability, disorganized schizotypy, and retrospective self-reported mind wandering changed as more probes were used to assess TUT rate. TUT rates were remarkably similar across increasing probe-bin sizes and zero-order correlations within and between tasks stabilized at 8–10 probes; moreover, TUT-rate correlations with other latent variables stabilized at about 8 thought probes. Our provisional recommendation (with caveats) is that researchers may use as few as 8 thought probes in prototypical cognitive tasks to gain reliable and valid information about individual differences in TUT rate.