Cognitive profile analysis in school psychology: History, issues, and continued concerns

The cognitive assessment course: Two decades later

“…Sotelo-Dynega and Dixon (2014) reported that nearly 70% of their participants continue to follow interpretive approaches learned during their course on cognitive testing. The administration and interpretation of cognitive assessment is not trivial to the role of school psychologists, nor is it without diversity of approaches (e.g., Benson et al, 2019;Canivez, 2013;Sattler, 2018) or controversy (see McGill, Dombrowski, & Canivez, 2018). Thus, it is crucial to examine the content, requirements, and testing practices taught in school psychology graduate training programs.…”

“…Items were initially generated with permission (V. Alfonso, personal communication, September 4, 2018) from past survey research (Alfonso et al, 2000), then updated based on recent review articles (e.g., Oakland & Wechsler, 2016), related research (e.g., Benson et al, 2019;Sotelo-Dynega & Dixon, 2014), and recent textbooks (e.g., Dombrowski, 2014;Hunsley & Mash, 2018;Kranzler & Floyd, 2013). Novel items related to theory and interpretation strategies were developed based on a review of related articles (e.g., McGill et al, 2018;Sotelo-Dynega & Dixon, 2014), textbook chapters from key authors (e.g., Canivez, 2013;Fiorello & Wycoff, 2018;Flanagan & McDonough, 2018;Sattler, 2018), and textbooks on interpretation procedures (e.g., Kaufman, Raiford, & Coalson, 2016). The survey was then reviewed by five content experts (i.e., five faculty members from four separate school psychology programs with experience teaching cognitive assessment) who provided feedback; items were added, revised, and deleted based on feedback.…”

“…Moreover, these findings remain partially consistent with Sattler's (2018) successive levels approach and Kaufman's intelligent testing approach (e.g., Kaufman, Raiford, & Coalson, 2016). While the interpretation of subtests and various methods for comparing subtests has been largely eschewed in the empirical literature (see McGill et al, 2018), many instructors continue to provide instruction about these methodsthough they rank lower in importance, on average, than other score types. This pattern is consistent with our finding that the emphasis on subtests has decreased since 1996 (Alfonso et al, 2000).…”

Difference score reliabilities within the RIAS‐2 and WISC‐V

“…When that data is unreliable, there is a higher risk for data to suggest a false positive or false negative, potentially misinforming IEP teams during the decision making process. Due to inconsistencies across age ranges and across tests, the necessity to individually calculate the confidence interval of each difference score (Charter, 1999;Charter & Feldt, 2009), and the poor diagnostic and treatment utility offered by profile analysis methods in general (Canivez, 2013;John H Kranzler, Floyd, Benson, Zaboski, & Thibodaux, 2016;McGill, 2018;McGill et al, 2018;McGill et al, 2016;Miciak, Fletcher, Stuebing, Vaughn, & Tolar, 2014;Watkins, 2003), clinicians will likely make more reliable clinical decisions if they avoid the use of difference scores. That said, difference scores, when used, should be interpreted with caution and only under the circumstances prescribed by Charter and colleagues (1999;.…”

“…However, given that test publishers do not provide the standard deviation and reliability of all scores-especially difference scores-from their instruments, estimating an appropriate CI width for difference scores is challenging As difference scores are typically established through a simple-difference procedure (i.e., subtracting one score from another), the errors of both scores (i.e., the contrast scores) will have a cumulative effect (Glass, Ryan, & Charter, 2010;McGill et al, 2018). Two essential components of a good difference score are (1) high contrast score reliability and (2) low-tomoderate intercorrelation between the contrasted scores.…”