Visual comparison of two data sets: Do people use the means and the variability?

Abstract

In our everyday lives, we are required to make decisions based upon our statistical intuitions. Often, these involve the comparison of two groups, such as luxury versus family cars and their suitability. Research has shown that the mean difference affects judgements where two sets of data are compared, but the variability of the data has only a minor influence, if any at all. However, prior research has tended to present raw data as simple lists of values. Here, we investigated whether displaying data visually, in the form of parallel dot plots, would lead viewers to incorporate variability information. In Experiment 1, we asked a large sample of people to compare two fictional groups (children who drank 'Brain Juice' versus water) in a one-shot design, where only a single comparison was made. Our results confirmed that only the mean difference between the groups predicted subsequent judgements of how much they differed, in line with previous work using lists of numbers. In Experiment 2, we asked each participant to make multiple comparisons, with both the mean difference and the pooled standard deviation varying across data sets they were shown. Here, we found that both sources of information were correctly incorporated when making responses. Taken together, we suggest that increasing the salience of variability information, through manipulating this factor across items seen, encourages viewers to consider this in their judgements. Such findings may have useful applications for best practices when teaching difficult concepts like sampling variation.Keywords: informal inferential reasoning, comparing groups, mean difference, pooled standard deviation, variability When deciding whether two groups are different on some measure, one of the most important concepts to understand is the mean or "average". Indeed, many teachers have focussed on determining the best ways to convey this idea to students at an early age, both through calculation and visual impression (Gal, 1995;Watson & Moritz, 1998). Even in adulthood, we are often presented with mean values in newspapers or television adverts (e.g., comparing the miles per gallon of two car models or the battery life of two smartphones) and expected to decide whether there is a meaningful difference. Problematically, research suggests that we place more weight than we should on these types of average ratings (e.g., de Langhe, Fernbach, & Lichtenstein, 2016). Statistically, information about the means alone is insufficient for making such decisions. One also requires knowledge of the variances (or some other measure of the "spread" of the two groups) in order to determine the size and importance of any difference. Unfortunately, understanding the concept of sampling distributions is both difficult for people (delMas, Garfield, &Chance, 1999) and under-researched (Meletiou, 2000). Journal of Numerical Cognition jnc.psychopen.eu | 2363-8761For several years, studies have investigated what has been termed 'informal inferential reasoning', where judgements are made...

Statistical Reasoning of Variability in the Narrative Perspective of Students

“…In that case, the ability of students in the group is more homogeneous, or it can be said that the students' ability is not much different from one student to another student. Variability can be used to choose one better group of several groups of data (Amaro and Sánchez, 2019; Kramer et al, 2017). For this reason, researchers assess the narrative proposed by subjects who chose group A can do the correct statistical reasoning using a narrative that follows the rules of variability.…”

Variable Biases: A Study of Scientists’ Interpretation of Plot Types Commonly Used in Scientific Communication

“…The dot, violin and density plots allowed participants to infer this information but did not mark it explicitly. While participants can summarize information that is not explicitly marked (e.g., [11,21]), explicit marking may lead to easier or more effective comprehension of that underlying element. The explicit markings on the bar plots may have provided more support to the participants for this task.…”

Measuring Categorical Perception in Color-Coded Scatterplots

“…Scatterplots have been studied across a range of tasks (see Sarikaya & Gleicher [62] for a survey). We employed a relative mean judgment task as applied in previous studies [13,23,31]. As in Gleicher et al [13], we asked participants to estimate the category with the highest average y-value.…”