Markov decision processes for train run curve optimization

Abstract

We propose three computationally efficient methods for finding optimal run curves of electrical trains, all based on the idea of approximating the continuous dynamics of a moving train by a Markov Decision Process (MDP) model. Deterministic continuous train dynamics are converted to stochastic transitions on a discrete model by observing the similarity between the properties of convex combinations and those of probability mass functions. The resulting MDP uses barycentric coordinates to effectively represent the cost-to-go of the approximated optimal control problem. One of the three solution methods uses equal distance steps, as opposed to the usual equal-time steps, to avoidself transitions of the MDP, which allows very fast computation of the cost-to-go in one pass only. IEEE International Conference on Electrical Systems for Aircraft, Railway and Ship Propulsion (ESARS)This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Abstract-We propose three computationally efficient methods for finding optimal run curves of electrical trains, all based on the idea of approximating the continuous dynamics of a moving train by a Markov Decision Process (MDP) model. Deterministic continuous train dynamics are converted to stochastic transitions on a discrete model by observing the similarity between the properties of convex combinations and those of probability mass functions. The resulting MDP uses barycentric coordinates to effectively represent the cost-to-go of the approximated optimal control problem. One of the three solution methods uses equaldistance steps, as opposed to the usual equal-time steps, to avoid self transitions of the MDP, which allows very fast computation of the cost-to-go in one pass only.

Optimization of Energy-Efficient Speed Profile for Electrified Vehicles

“…Optimal speed profiles have been researched for different vehicles and transportations including off-road vehicles [20], and vehicles moving on fixed routes, such as trains [21], buses [22], or race cars [23], [24]. [33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52]. The solutions of which can provide a vehicle speed profile that decreases the amount of energy consumed over the trip.…”

Optimization of thermal modes and cooling systems of the induction traction engines of trams

“…Optimization of the processes of designing the traction engines, common at the enterprises of leading electrotechnical manufacturers, makes it possible to create traction engines optimal by efficiency [2,12]. However, the operating modes of the electric rolling stock, which moves at different speeds and at different loading regimes [13], considerably reduce the overall efficiency of electric rolling stock [14]. Determining motion modes optimal by energy consumption enables to increase the efficiency of cooling system of traction engines, as well as of electric rolling stock in general [15].…”

“…Therefore, to determine the thermal state of a traction engine, it is necessary to consider thermal load over the entire period of operation [15]. When using a traction drive, common modes are overshooting [13] and mechanical (pneumatic) braking [13]. Conversion of energy under these modes does not occur in a traction engine [19].…”

Synthesis of Pontryagin's Maximum Principle Analysis for Speed Profile Optimization of All-Electric Vehicles

“…Eco-driving or energyefficient driving is the term given to the idea of determining the speed trajectory that minimizes the vehicle energy consumption under final time and distance constraints. This problem has been addressed as an optimal control problem (OCP) with related work done by researchers [3][4][5][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23], the solutions of which can provide a vehicle speed profile that decreases the amount of energy consumed over the trip.…”

“…In solving the trajectory optimization problem, numerical approaches such as genetic algorithm [7,14,16], ant colony algorithm [16], sequential quadratic programming [13], nonlinear programming [24], and dynamic programming (DP) [15][16][17]23,25] have been used for nonlinear characteristics, such as varying speed limits, resistive forces, and signal-phase timing. Among these, DP has been widely and extensively used because it can find a global optimum even for nonlinear systems with nonlinear constraints.…”