Do skilled decision-makers plan further into the future than novices? This question has been investigated for almost 75 years, traditionally by studying expert players in complex board games like chess. However, the complexity of these games poses a barrier to detailed modeling of human behavior. Conversely, common planning tasks in cognitive science are often lower-complexity and impose a ceiling for the depth to which any player can plan. Here, we investigate expertise by studying decision-making in a board game which is at the limit of complexity that can be precisely modeled using state-of-the-art statistical techniques, and which has ample opportunity for skilled players to plan deeply. We find robust evidence for increased planning depth with expertise in both laboratory and large-scale naturalistic data.

When developing models in cognitive science, researchers typically start with their own intuitions about human behavior in a given task and then build in mechanisms that explain additional aspects of the data. This refinement step is often hindered by how difficult it is to distinguish the unpredictable randomness of people’s decisions from meaningful deviations between those decisions and the model. One solution for this problem is to compare the model against deep neural networks trained on behavioral data, which can detect almost any pattern given sufficient data. Here, we apply this method to the domain of planning with a heuristic search model for human play in 4-in-a-row, a combinatorial game where participants think multiple steps into the future. Using a data set consisting of 10,874,547 games, we train deep neural networks to predict human moves and find that they accurately do so while capturing meaningful patterns in the data. Thus, deviations between the model and the best network allow us to identify opportunities for model improvement despite starting with a model that has undergone substantial testing in previous work. Based on this analysis, we add three extensions to the model that range from a simple opening bias to specific adjustments regarding endgame planning. Overall, our work demonstrates the advantages of model comparison with a high-performance deep neural network as well as the feasibility of scaling cognitive models to massive data sets for systematically investigating the processes underlying human sequential decision-making.

No abstract