Automated text-based analysis for decision-making research

Abstract

We present results from a study on constructing and evaluating a support tool for the extraction of patterns in distributed decision making processes, based on design criteria elicited from a study on the work process involved in studying such decision making. Specifically, we devised and evaluated an analysis tool for C 2 researchers who study simulated decision-making scenarios for command teams. The analysis tool used text clustering as an underlying pattern extraction technique, and was evaluated together with C 2 researchers in a workshop to establish whether the design criteria were valid and the approach taken with the analysis tool was sound.Design criteria elicited from an earlier study with researchers (open-endedness and transparency) were highly consistent with the results from the workshop. Specifically, evaluation results indicate that successful deployment of advanced analysis tools require that tools can treat multiple data sources and offer rich opportunities for manipulation and interaction (open-endedness) and careful design of visual presentations and explanations of the techniques used (transparency). Finally, the results point to the high relevance and promise of using text clustering as a support for analysis of C 2 data.