Elicitation by Critiquing: An Exploratory Study

Abstract

Knowledge elicitation methods uncover information about how a practitioner works in a field. However, these methods have the challenges of grounding in context, accessibility to experts and tasks, being laborious and lime consuming, and repeatability. This study investigated a critiquing methodology's ability to address these challenges while still being authentic and generative. The domain investigated was military intelligence analysis, a field that experiences a constant turnover of workers. To maintain the highest level of productivity, the expertise of their miliintelligence analysis practitioners must be understd. A study was done involving a novice performing a basic analysis task. Then, six experts with various backgrounds critiqued the novice's process. The results suggest that the critiquing method addresses the challenges of knowledge elicitation methods while being authentic and generative. The specific guiding question for addressing both issues was 'Can the critiquing method be used to help unveil expertise in intelligence analysis?'