From Natural Language Models to Cognitive Model Validation: A Theoretical Framework

Abstract

We propose a novel technique that uses individuals' verbal descriptions of their problem-solving strategies to validate psychological processes assumed by computational cognitive models. We capitalize on recent advances in Natural Language Processing models (NLP), in particular their context-sensitivity, to classify participants’ unstructured verbal descriptions. We illustrate our approach with an experiment examining how people integrate social and private information when making risky decisions. We contrast NLP model outputs derived from participants’ verbal descriptions with the assumptions underlying a computational model fit to the behavioural data. We discuss ways to refine and improve this technique, and argue that verbal descriptions are a valuable and under-utilized source of data for holding cognitive models accountable to their psychological assumptions.