The Communicability of Graphical Alternatives to Tabular Displays of Statistical Simulation Studies

Abstract

Simulation studies are often used to assess the frequency properties and optimality of statistical methods. They are typically reported in tables, which may contain hundreds of figures to be contrasted over multiple dimensions. To assess the degree to which these tables are fit for purpose, we performed a randomised cross-over experiment in which statisticians were asked to extract information from (i) such a table sourced from the literature and (ii) a graphical adaptation designed by the authors, and were timed and assessed for accuracy. We developed hierarchical models accounting for differences between individuals of different experience levels (under- and post-graduate), within experience levels, and between different table-graph pairs. In our experiment, information could be extracted quicker and, for less experienced participants, more accurately from graphical presentations than tabular displays. We also performed a literature review to assess the prevalence of hard-to-interpret design features in tables of simulation studies in three popular statistics journals, finding that many are presented innumerately. We recommend simulation studies be presented in graphical form.

Statistician Heal Thyself: Have We Lost the Plot?

“…Gelman and others (Cleveland, 1984;Gelman et al, 2002;Kastellec and Leoni, 2007) have identified a need for greater use of graphs in statistics and allied disciplines literature. Cook and Teo (2011) recommended the use of graphs over tables based on the speed and accuracy with which information about statistical simulation studies could be decoded. However, the skeptics or "table people" (Friendly and Kwan, 2011) are unlikely to be convinced of the power of graphics without examples of graphical excellence.…”