Neuropsychological tests of the future: How do we get there from here?

Abstract

Objective: This article reviews current approaches to neuropsychological assessment, identifies opportunities for development of new methods using modern psychometric theory and advances in technology, and suggests a transition path that promotes application of novel methods without sacrificing validity. Methods: Theoretical/state-of-the-art review. Conclusions: Clinical neuropsychological assessment today does not reflect advances in neuroscience, modern psychometrics, or technology. Major opportunities for improving practice include both psychometric and technological strategies. Modern psychometric approaches including item response theory (IRT) enable: linking procedures that can place different measures on common scales; adaptive testing algorithms that can dramatically increase efficiency of assessment; examination of differential item functioning (DIF) to detect measures that behave differently in different groups; and person fit statistics to detect aberrant patterns of responding of high value for performance validity testing. Opportunities to introduce novel technologies include computerized adaptive testing, Web-based assessment, healthcare-and bio-informatics strategies, mobile platforms, wearables, and the "internet-of-things." To overcome inertia in current practices, new methods must satisfy requirements for back-compatibility with legacy instrumentation, enabling us to leverage the wealth of validity data already accrued for classic procedures. A path to achieve these goals involves creation of a global network to aggregate item-level data into a shared repository that will enable modern psychometric analyses to refine existing methods, and serve as a platform to evolve novel assessment strategies, which over time can revolutionize neuropsychological assessment practices worldwide .

Digital neuropsychological assessment: Feasibility and applicability in patients with acquired brain injury

“…Norms are ideally updated regularly (Germine et al, 2019). However, many paper-and-pencil tests exist for decades and test performances are interpreted using norms from studies that were conducted several decades ago (in this study ranging from the year 1993-2012) (Bilder & Reise, 2019;Dickinson & Hiscock, 2011). General experiences of a population change over time and highly affect test performance, also known as the Flynn effect (Dickinson & Hiscock, 2011).…”

“…There are, however, several aspects that compromise the usability of computerized test batteries in clinical practice (Bauer et al, 2012;Bilder & Reise, 2019;Schlegel & Gilliland, 2007). For instance, introducing new tests in clinical practice requires clinicians to invest time in learning the structures, instructions and underlying constructs of the tests.…”

“…Furthermore, some test batteries allow self-administration with minimal interaction between the clinician and patient. Important behavioral observations, such as fatigue or unexpected distractors, are therefore lost (Bilder & Reise, 2019;Harvey, 2012;Kaplan, 1988;Witt et al, 2013). Finally, an individual's familiarity with a response device (e.g., keyboard, computer mouse, joystick or touch-screen devices) may affect test performance (Germine et al, 2019).…”

Neuropsicologia no Brasil: passado, presente e futuro

“…Para R. Bilder e S. Reise (2019), as principais oportunidades para melhoria da prática neuropsicológica envolvem estratégias psicométricas e tecnológicas. Os autores destacam a importância de abordagens psicométricas, tais como a Teoria de Resposta ao Item (TRI), pois estas permitem a conexão entre procedimentos a partir da inclusão de diferentes medidas em escalas comuns; o uso de algoritmos de testes adaptativos que poderiam aumentar a eficiência da avaliação; o uso do exame de funcionamento diferencial dos itens (DIF) para detectar medidas que se comportam diferentemente em grupos distintos (Bilder & Reise, 2019). Por sua vez, o uso da tecnologia desponta igualmente como um desafio e uma oportunidade.…”

How to Resolve Controversies in Cognitive Training: Let the Data Speak!

“…There are multiple ways forward: Increase use of common data elements: If more studies used shared outcome measures mega- and meta-analyses would be clearer, sources of heterogeneity easier to identify, and potential confounds easier to eliminate. Establish psychometric criteria for domain definitions and boundaries : If investigators wish to declare “domains,” these constructs should be specified in advance and satisfy psychometric standards for reliability, precision and sensitivity across the range of the relevant trait. We could also specify statistical boundaries to quantify what is “near” and what is “far”, based on shared variance or other psychometric distance parameters. Share data openly : burgeoning opportunities for open science contributions, and increased scope of NIMH Data Archives that accommodate cognitive measures, provide the basis for data-pooling and shared analysis. Apply modern psychometric and factor analytic strategies : To determine the commonalities among tests and enable more robust pooling of data across studies, we should use modern psychometric approaches to test-linking, harmonization and alignment, that allow estimation of individual differences on shared traits even though different tests may be used (8). …”