A deep reinforcement learning approach to mountain railway alignment optimization

Abstract

The design and planning of railway alignments is the dominant task in railway construction. However, it is difficult to achieve self-learning and learning from human experience with manual as well as automated design methods. Also, many existing approaches require predefined numbers of horizontal points of intersection or vertical points of intersection as input. To address these issues, this study employs deep reinforcement learning (DRL) to optimize mountainous railway alignments with the goal of minimizing construction costs. First, in the DRL model, the state of the railway alignment optimization environment is determined, and the action and reward function of the optimization agent are defined along with the corresponding alignment constraints. Second, we integrate a recent DRL algorithm called the deep deterministic policy gradient with optional human experience to obtain the final optimized railway alignment, and the influence of human experience is demonstrated through a sensitivity analysis. Finally, this methodology is applied to a real-world case study in a mountainous region, and the results verify that the DRL approach used here can automatically explore and optimize the railway alignment, decreasing the construction cost by 17.65% and 7.98%, compared with the manual alignment and with the results of a method based on the distance transform, respectively, while satisfying various alignment constraints. INTRODUCTIONThe planning and design of railway alignments is not only the foundation of railway construction but also an extensive and systematic task. The direction of a railway alignment directly affects the difficulty, cost, and safety of the railway construction and operation. Therefore, the final railway alignment design should not only consider a series of natural factors such as geology and topography in the © 2021 Computer-Aided Civil and Infrastructure Engineering railway area but also satisfy other constraints, including those regarding the existing railway, historical sites, and environmental protection zones in the target area. Overall, pathfinding for a railway alignment is an optimization and decision-making problem involving many restrictive factors (Li et al., 2013). Traditional railway path planning is performed manually. Based on work experience and accumulated knowledge, designers analyze, evaluate, and compare multiple Comput Aided Civ Inf. 2022;37:73-92.wileyonlinelibrary.com/journal/mice ing requirements, thus ensuring that the horizontal alignment satisfies the constraints in the horizontal plane. In this study, we set 𝑅 𝑖 as a fixed value, which should exceed the minimum allowed value of 600 m (specified in Table 1), to fit the horizontal circular curve.

Exploring and exploiting ant colony optimization algorithm for vertical highway alignment development

“…At present, the proposed model requires a well‐established horizontal alignment. The proposed model can be integrated with horizontal alignment development models (Bosurgi & D'andrea, 2012; Gao et al., 2022; Sushma & Maji, 2020; Vázquez‐Méndez et al., 2021a) to achieve a holistic highway alignment development model. It will be useful in the automation of highway development.…”

Joint power distribution and charging network design for electrified mobility with user equilibrium decisions

“…Other future directions of the proposed study follow. Additional benchmarking methods can be used to evaluate the proposed methodology, such as decomposition techniques (e.g., Hajibabai & Saha, 2019; Mirheli et al., 2019; Mohebifard & Hajbabaie, 2021) as well as machine learning approaches (e.g., Alam et al., 2020; Chen et al., 2020, 2021; Gao et al., 2021; Mirheli et al., 2018; Nabian & Meidani, 2018; Pereira et al., 2020; Rafiei & Adeli, 2017). Besides, EV user charging schedules and pricing schemes can be integrated into the PDN design.…”

Modeling side slopes in vertical alignment resource road construction using convex optimization

“…Many efficient models have been proposed for vertical alignment optimization incorporating various interesting techniques: various heuristic approaches (e.g., Akay, 2003; Goktepe et al., 2009; Jong & Schonfeld, 2003; Lee & Cheng, 2001; Li et al., 2017; Song et al., 2021; Vázquez‐Méndez et al., 2021), deep learning techniques (e.g., Gao et al., 2022), dynamic programming (e.g., Fwa, 1989; Goh et al., 1988; Goktepe et al., 2005; Li et al., 2013), linear or mixed integer linear programming (MILP; e.g., Easa, 1988; Hare et al., 2011, 2015; Koch & Lucet, 2010; Moreb, 1996, 2009; Moreb & Aljohani, 2004), and other methods (e.g., Ozkan et al., 2021).…”