Für autonome Fahrzeuge stellt die Pfadfolgeregelung eine Schlüsselfunktion dar. Die Pfadfolgeregelung steuert hierbei Antrieb, Lenkung und Bremse derart, dass das Fahrzeug einem geometrischen Pfad mit einer Referenzgeschwindigkeit folgt. Für die Auslegung von leistungsfähigen modellbasierten Pfadfolgereglern wird ein ausreichend genaues Synthesemodell des Fahrzeuges benötigt. Der Entwurf, die Parametrierung und das Testen von modellbasierten Pfadfolgereglern, sowie das Ableiten eines Synthesemodells ist allerdings eine zeitaufwändige Aufgabe. In der klassischen Regelungstechnik werden deshalb vermehrt Reinforcement Learning (RL) Methoden angewandt, um Regelungsprobleme ohne Synthesemodell, nur mit Hilfe von hochgenauen Simulationsmodellen zu lösen. Um den Einsatz von RL Methoden auf das Pfadfolgeproblem zu untersuchen, wird in diesem Beitrag die Anwendung am Beispiel des überaktuierten robotischen Fahrzeuges ROboMObil des DLRs vorgestellt. Erste Simulationsergebnisse zeigen, dass RL basierte Pfadfolgeregler auf dem Trainingspfad ein ähnlich gutes Folgeverhalten aufweisen, wie modellbasierte Pfadfolgeregler. Die RL basierten Regler erzielen dabei auch auf neuen und unbekannten Pfaden gute Ergebnisse.

High-level modeling languages facilitate system modeling and the development of control systems. This is mainly achieved by the automated handling of differential algebraic equations which describe the dynamics of the modeled systems across different physical domains. A wide selection of model libraries provides additional support to the modeling process. Nevertheless, deployment on embedded targets poses a challenge and usually requires manual modification and reimplementation of the control system. The novel proposed eFMI Standard (Functional Mock-up Interface for embedded systems) introduces a workflow and an automated toolchain to simplify the deployment of model-based control systems on embedded targets. This contribution describes the application and verification of the eFMI workflow using a vertical dynamics control problem with an automotive application as an example. The workflow is exemplified by a control system design process which is supported by the a-causal, multi-physical, high-level modeling language Modelica. In this process, the eFMI toolchain is applied to a model-based controller for semi-active dampers and demonstrated using an eFMI-based nonlinear prediction model within a nonlinear Kalman filter. The generated code was successfully tested in different validation steps on the dedicated embedded system. Additionally, tests with a low-volume production electronic control unit (ECU) in a series-produced car demonstrated the correct execution of the controller code under real-world conditions. The novelty of our approach is that it automatically derives an embedded software solution from a high-level multi-physical model with standardized eFMI methodology and tooling. We present one of the first full application scenarios (covering all aspects ranging from multi-physical modeling up to embedded target deployment) of the new eFMI tooling.

In this contribution we propose a reinforcement learning-based controller which is able to solve the path following problem for vehicles with significant delay in the drivetrain. To efficiently train the controller, a control-oriented simulation model for a vehicle with combustion engine, automatic gear box and hydraulic brake system has been developed. In addition, to enhance the reinforcement learning-based controller, we have introduced preview information in the feedback state to better deal with the delays. We present our approach of designing a reward function which enables the reinforcement learning-based controller to solve the problem. The controller is trained using the Soft Actor-Critic algorithm by incorporating the developed simulation model. Finally, the performance and robustness is evaluated in simulation. Our controller is able to follow an unseen path and is robust against variations in the vehicle parameters, in our case an additional payload.

Traffic congestion and the occurrence of traffic accidents are problems that can be mitigated by applying cooperative adaptive cruise control (CACC). In this work, we used deep reinforcement learning for CACC and assessed its potential to outperform model-based methods. The trade-off between distance-error minimization and energy consumption minimization whilst still ensuring operational safety was investigated. Alongside a string stability condition, robustness against burst errors in communication also was incorporated, and the effect of preview information was assessed. The controllers were trained using the proximal policy optimization algorithm. A validation by comparison with a model-based controller was performed. The performance of the trained controllers was verified with respect to the mean energy consumption and the root mean squared distance error. In our evaluation scenarios, the learning-based controllers reduced energy consumption in comparison to the model-based controller by 17.9% on average.

No abstract