Evaluating Staggered Working Hours Using a Multi-Agent-Based Q-Learning Model

Abstract

Staggered working hours has the potential to alleviate excessive demands on urban transport networks during the morning and afternoon peak hours and influence the travel behavior of individuals by affecting their activity schedules and reducing their commuting times. This study proposes a multi-agent-based Q-learning algorithm for evaluating the influence of staggered work hours by simulating travelers’ time and location choices in their activity patterns. Interactions among multiple travelers were also considered. Various types of agents were identified based on real activity–travel data for a mid-sized city in China. Reward functions based on time and location information were constructed using Origin–Destination (OD) survey data to simulate individuals’ temporal and spatial choices simultaneously. Interactions among individuals were then described by introducing a road impedance function to formulate a dynamic environment in which one traveler’s decisions influence the decisions of other travelers. Lastly, by applying the Q-learning algorithm, individuals’ activity–travel patterns under staggered working hours were simulated. Based on the simulation results, the effects of staggered working hours were evaluated on both a macroscopic level, at which the space–time distribution of the traffic volume in the network was determined, and a microscopic level, at which the timing of individuals’ leisure activities and their daily household commuting costs were determined. Based on the simulation results and experimental tests, an optimal scheme for staggering working hours was developed.

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

“…To study the marginal value theorem (MVT), Miller et al [98] implemented four algorithms in a large-scale avian foraging model, including online MVT (OMVT), extended OMVT (XOMVT), reinforcementlearning MVT (RLMVT), and extended RLMVT (XRLMT), and they found that RL algorithms (RLMVT/XRLMVT) performed far better for approximating marginal values than continuous estimation or online algorithms (OMVT/XOMVT). Yang et al [99] also applied the Q-learning algorithm to model the complex time-space choice behaviors of agents in an activity-travel scheduling process.…”

Modeling the Peak-Period Bus Commuting Behavior with Staggered Work Hours Using a Regret-Minimizing Learning Method

“…It has been proved that agent-based simulation technology is efective, fexible, and expansible in trafc system modelling [20,21]. Yang et al [22] utilized a multiagent-based Q-learning algorithm for evaluating the infuence of SWH policy by simulating travelers' time and location choices in their activity patterns. Xie et al [23] simulated commuter departure time choices based on the BM reinforcement learning model in a many-to-one bus transit scenario.…”

Solving Traffic Congestion From the Demand Side

“…It is difficult to find a tool for evaluating short-term, medium-term and long-term effects of a potential TDM strategy. Among those offering such tools, [50] proposed a multi-agent based Q-learning algorithm to evaluate the effects of In empirical analysis, the regression analysis, or its variants, is still a technique used most. To extract origins and destinations, search for matched trips, etc.…”