“…While validity and reliability are occasionally presented within and across groups (Łuniewska et al, 2016;Moors et al, 2013;Schröder et al, 2012), intra-rater reliability, that is, whether the rater is consistent in his or her ratings, and the validity of each individual's ratings is seldom reported in studies obtaining subjective ratings of AoA (Alario & Ferrand, 1999;Bird et al, 2001;Bonin, Peereman, Malardier, Méot, & Chalard, 2003;Cortese & Khanna, 2007;Ferrand et al, 2008;Łuniewska et al, 2016;Moors et al, 2013;Raman et al, 2014;Stadthagen-Gonzalez & Davis, 2006;Zevin & Seidenberg, 2004). Exploring the validity of the instruments (i.e., the individual raters) should be a high priority, in particular when the number of raters is limited, often around 18-35 individuals (Alario & Ferrand, 1999;Brysbaert et al, 2014;Cortese & Khanna, 2007;Della Rosa, Catricalà, Vigliocco, & Cappa, 2010;Ferrand et al, 2008;Łuniewska et al, 2016;Moors et al, 2013;Stadthagen-Gonzalez & Davis, 2006). In their large web-based study, Kuperman, Stadthagen-Gonzalez, and Brysbaert (2012) did explore the validity of each individual rater, by correlating their subjective AoA ratings with previously obtained AoA norms (Stadthagen-Gonzalez & Davis, 2006).…”