Understanding Human Decision Processes: Inferring Decision Strategies From Behavioral Data

Abstract

This work investigates a method to infer and classify decision strategies from human behavior, with the goal of improving human-agent team performance by providing AI-based decision support systems with knowledge about their human teammate. First, an experiment was designed to mimic a realistic emergency preparedness scenario in which the test participants were tasked with allocating resources into 1 of 100 possible locations based on a variety of dynamic visual heat maps. Simple participant behavioral data, such as the frequency and duration of information access, were recorded in real time for each participant. The data were examined using a partial least squares regression to identify the participants’ likely decision strategy, that is, which heat maps they relied upon the most. The behavioral data were then used to train a random forest classifier, which was shown to be highly accurate in classifying the decision strategy of new participants. This approach presents an opportunity to give AI systems the ability to accurately model the human decision-making process in real time, enabling the creation of proactive decision support systems and improving overall human-agent teaming.

Development of Mental Models in Decision-Making Tasks

“…A previously established approach, Partial Least Squares Regression (PLS-R), was used for strategy identification in Walsh and Feigh (2022a, 2022b). PLS-R is well suited for this task domain because it can be used when there are correlated independent variables.…”

“…VIP scores higher than 1 indicate the data information sources, i.e., attributes, that were used (1 being a wellestablished cutoff threshold for PLS-R independent variable reduction as supported by Akarachantachote, Chadcham, and Saithanu (2014)). For more information on this approach, refer to Walsh and Feigh (2022b).…”

“…The classification of decision models based on behavioral data is relevant in most cases because the data can be easily collected in most processes and the ground truth labels are generally unknown. In this study, we build upon Walsh and Feigh (2022b) by temporally extending the set of decision tasks from 10 updates to 30. At each update, the information sources are updated and participants must choose an optimal location based on the information available at that update.…”

“…In a previous study, Walsh and Feigh (2022b) proposed an objective method to assess the general model of DM using real-time, observable behavioral data. The classification of decision models based on behavioral data is relevant in most cases because the data can be easily collected in most processes and the ground truth labels are generally unknown.…”

Special Issue on Human-AI Teaming and Special Issue on AI in Healthcare

“…Several researchers extend our overall understanding of methods for improving human-AI teaming. Walsh and Feigh (2022) present an interesting analytic method through which an AI system can infer the decision strategy of a human teammate. The method uses machine learning to classify the decision strategy based on behavioral data collected from the human teammate.…”