Segregation dynamics with reinforcement learning and agent based modeling

Abstract

Societies are complex. properties of social systems can be explained by the interplay and weaving of individual actions. Rewards are key to understand people's choices and decisions. For instance, individual preferences of where to live may lead to the emergence of social segregation. In this paper, we combine Reinforcement Learning (RL) with Agent Based Modeling (ABM) in order to address the self-organizing dynamics of social segregation and explore the space of possibilities that emerge from considering different types of rewards. Our model promotes the creation of interdependencies and interactions among multiple agents of two different kinds that segregate from each other. For this purpose, agents use Deep Q-Networks to make decisions inspired on the rules of the Schelling Segregation model and rewards for interactions. Despite the segregation reward, our experiments show that spatial integration can be achieved by establishing interdependencies among agents of different kinds. They also reveal that segregated areas are more probable to host older people than diverse areas, which attract younger ones. Through this work, we show that the combination of RL and ABM can create an artificial environment for policy makers to observe potential and existing behaviors associated to rules of interactions and rewards. The recent availability of large datasets collected from various resources, such as digital transactions, location data and government census, is transforming the ways we study and understand social systems 1. Researchers and policy makers are able to observe and model social interactions and dynamics in great detail, including the structure of friendship networks 2 , the behavior of cities 3 , politically polarized societies 4 , or the spread of information on social media 5. These studies show the behaviors present in the data but do not explore the space of possibilities that human dynamics may evolve to. Robust policies should consider mechanisms to respond to every type of events 6 , including those that are very rare 7. Therefore it is crucial to develop simulation environments such that potentially unobserved social dynamics can be assessed empirically. Agent Based Modeling (ABM) is a generative approach to study natural phenomena based on the interaction of individuals 8 in social, physical and biological systems 9. These models show how different types of individual behavior give rise to emergent macroscopic regularities 10,11 with forecasting capabilities 12. Applications to social systems include the emergence of wealth distributions 13 , new political actors 14 , multipolarity in interstate systems 15 , and cultural differentiation 16 , among other applications 9. ABM allows testing core sociological theories against simulations 13 with emphasis on heterogeneous, autonomous actors with bounded, spatial information 17. They provide a framework to understand complex behaviors like those of economic systems 18,19 , as well as individual 20 and organizational 21,22 decision making processes. These ...

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

“…In 2018, Jang et al [100] applied one of the DRL algorithmsdeep Q-network (DQN) to traffic simulation to build an effective ABM. In 2019, Sert et al [101] also applied DQN and ABM-based approaches to build an artificial environment model that can signify the segregation dynamic of a living human population. In these DQN-based approaches, the DNN was generally used as a function approximator to estimate the Q value, therefore avoiding the use of the Q-table.…”

Remittances, ethnic diversity, and entrepreneurship in developing countries

“…For example, social psychology research shows that emphasizing superordinate goals promotes trust and reduces tensions (Sherif, 1958;Gaertner et al, 2000). Moreover, Sert at al. (2020) have recently demonstrated that creating interdependencies among the agents of different kinds leads to increased interaction and cooperation.…”

Optimal Agent Framework: A Novel, Cost-Effective Model Articulation to Fill the Integration Gap between Agent-Based Modeling and Decision-Making

“…For resolving the computational cost of this method, researchers have proposed different methods. e intelligent agent method for building a new version of the Schelling segregation model [32] has been used in [75]. ey combined RL and ABM to build a model 8 Complexity in which agents make decisions using an RL algorithm called Deep-Q network.…”