Verbal Reports as Data Revisited: Using Natural Language Models to Validate Cognitive Models

Abstract

We propose a novel technique that uses individuals’ verbal reports to validate psychological processes assumed by computational cognitive models. We capitalize on recent advances in Natural Language Processing (NLP) models, especially their context-sensitivity, and recommend using them to classify participants’ unstructured verbal descriptions of the strategies they use to perform tasks. This framework emphasizes that verbal descriptions are a valuable and under-utilized source of data for ensuring cognitive models align with their psychological assumptions. We argue that this framework can encompass a broad range of cognitive tasks, including problem solving, reasoning, memory, categorisation, and decision-making