Sources of measurement error in pediatric intelligence testing

Abstract

Pediatric intelligence tests, such as the Wechsler Intelligence Scale for Children, are commonly used diagnostic tools used in the process of diagnosing learning and behavior disabilities. Decisions concerning treatment are made based on the results of these tests and they are frequently used in educational and vocational contexts for important decisions that impact persons’ academic or professional lives. Research has however shown that important errors may occur despite the application of validation processes and adherence to quality criteria for psychometric tests. At the same time this evidence seems not to be pervasively acknowledged in psychological practice and research. In this article, I will showcase research that places attention on sources of measurement error in pediatric intelligence testing, discuss a process-performance approach to measurement in intelligence testing, and propose the “pretest methods,” methods stemming from the field of survey methodology commonly used in questionnaire construction, as a method to help address the problem of sources of measurement error in pediatric intelligence testing and improve the development of these intelligence tests.

A Narrative Review of the Sociotechnical Landscape and Potential of Computer-Assisted Dynamic Assessment for Children with Communication Support Needs

“…A simple example might be left-to-right cognitive biases determined by western orthography, and therefore not reflected in those from Hebrew or Arabic backgrounds. Similarly, children who experience anxiety in unfamiliar situations, have attention, behavioural, or sensory processing difficulties or motor impairments, will struggle to achieve within the context of a time-limited, one-off evaluation-in particular, when that test is administered by a stranger [12]. In the case of children with CSN who may use augmentative and alternative tools and strategies to communicate, there is clearly an additional layer of complexity not conducive to a brisk psychometric testing procedure [12,32].…”

“…Similarly, children who experience anxiety in unfamiliar situations, have attention, behavioural, or sensory processing difficulties or motor impairments, will struggle to achieve within the context of a time-limited, one-off evaluation-in particular, when that test is administered by a stranger [12]. In the case of children with CSN who may use augmentative and alternative tools and strategies to communicate, there is clearly an additional layer of complexity not conducive to a brisk psychometric testing procedure [12,32]. These real-world issues represent a significant obstacle to the implementation of fair testing practices and the capture of accurate results that will enable educators to tailor the most effective pedagogical programmes for a notably vulnerable demographic.…”

“…For many (additional examples include minorities from culturally/linguistically diverse populations [10])-and in particular the focal population of this paper, young emerging aided communicators with communication support needs (CSN), who may be living with motor, cognitive, sensory and/or communicative impairments (for example, those with little or no natural speech and are consequently reliant on augmentative and alternative communication (AAC) solutions-addressed in more detail in Section 3.3.2 below-in their interactions with others)-this presumption can be flawed at best [11]. In fact, for an array of subjective reasons explored in more detail below, it skews the accuracy of such tests to the extent that the results are simply invalid as a predictor of learning potential-with implications for the child's pedagogical support that may prove deleterious and long lasting [12].…”

Intelligence tests and the individual: Unsolvable problems with validity and reliability

“…Apart from cognitive preparedness, children can also be more or less prepared for test taking on an emotional level. Test-anxiety or self-confidence, for example, have a great effect on test outcome (Torres Van Grinsven, 2022), and may be transferred from parents to children (Richardson, 2002). Weiss and Saklofske (2020) further point at the significant impact of SES, parental income, parental expectations, and academic monitoring on the IQ scores of children.…”

The Response Process in Psychometric (Intelligence) Testing and Validity

“…When diagnosing and treating learning and behavior disabilities, psychometric tests are commonly used diagnostic tools. Research has however shown that important errors may occur despite the application of validation processes and adherence to quality criteria for psychometric tests [1]. In a seminal work about the construction of questions for (standardized) questionnaires used in social and behavioral research, Foddy [2] presents the basic assumption in the science of questionnaire construction that question-answer behavior involves complex interrelationships between sociological, psychological and linguistic variables [2], which (my addition) results in a response process.…”