Towards Generalizing Sensorimotor Control Across Weather Conditions

Khan, Qadeer; Wenzel, Patrick; Cremers, Daniel; Leal-Taixé, Laura

doi:10.1109/iros40897.2019.8968451

Cited by 7 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The unit of measurement we use for online evaluation is the ratio on lane metric adapted from [14]. It gives the ratio of frames the ego-vehicle remains within its own driving lane to the total number of frames.…”

Section: Methodsmentioning

confidence: 99%

“…In the point clouds generated from dense depth maps, it may be hard to recognize relevant high-level features like road markings, traffic poles etc. These high level features tend to be important for the model to take the appropriate steering decisions [14]. Moreover, having redundant points in the point cloud which do not yield useful information for the vehicle control model would impose an unnecessary computational burden.…”

Section: A Point Cloud Generationmentioning

confidence: 99%

“…As can be observed, the performance of such a model trained with partially observable point clouds drops dramatically. Edge Filtered v. Full Point Cloud: The authors of [14] alluded to high level features such as lane markings, sidewalk/road intersections, barriers etc. being important for the vehicle control model to hold its driving lane.…”

Section: Single Trajectory V Multiple Generated Trajectoriesmentioning

confidence: 99%

See 2 more Smart Citations

Lateral Ego-Vehicle Control without Supervision using Point Clouds

Fabbri¹,

Khan²,

Cremers³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Existing vision based supervised approaches to lateral vehicle control are capable of directly mapping RGB images to the appropriate steering commands. However, they are prone to suffering from inadequate robustness in real world scenarios due to a lack of failure cases in the training data. In this paper, a framework for training a more robust and scalable model for lateral vehicle control is proposed. The framework only requires an unlabeled sequence of RGB images. The trained model takes a point cloud as input and predicts the lateral offset to a subsequent frame from which the steering angle is inferred. The frame poses are in turn obtained from visual odometry. The point cloud is conceived by projecting dense depth maps into 3D. An arbitrary number of additional trajectories from this point cloud can be generated during training. This is to increase the robustness of the model. Online experiments show that the performance of our method is superior to that of the supervised model.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: A Point Cloud Generationmentioning

confidence: 99%

Section: Single Trajectory V Multiple Generated Trajectoriesmentioning

confidence: 99%

See 1 more Smart Citation

Lateral Ego-Vehicle Control without Supervision using Point Clouds

Fabbri¹,

Khan²,

Cremers³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For e.g. a control model trained on images from a sunny weather condition would have difficulty controlling the vehicle in a rainy weather condition even though the domains may be the same [68]. Note that the entire code for training and conducting both offline along with online evaluation is contained in the fol-lowing repository: https://github.com/Dekai21/ Multi_Agent_Intersection.…”

Section: Domain Adaptationmentioning

confidence: 99%

Multi-Vehicle Trajectory Prediction at Intersections using State and Intention Information

Zhu¹,

Khan²,

Cremers³

2023

Preprint

View full text Add to dashboard Cite

Traditional approaches to prediction of future trajectory of road agents rely on knowing information about their past trajectory. This work rather relies only on having knowledge of the current state and intended direction to make predictions for multiple vehicles at intersections. Furthermore, message passing of this information between the vehicles provides each one of them a more holistic overview of the environment allowing for a more informed prediction. This is done by training a neural network which takes the state and intent of the multiple vehicles to predict their future trajectory. Using the intention as an input allows our approach to be extended to additionally control the multiple vehicles to drive towards desired paths. Experimental results demonstrate the robustness of our approach both in terms of trajectory prediction and vehicle control at intersections. The complete training and evaluation code for this work is available here: https:// github.com/Dekai21/Multi_Agent_Intersection.

show abstract

“…Nonetheless, our method for car following is robust against the remaining void regions caused by occlusion or discontinuities in the predicted depth map. [26] showed that the sensorimotor control model focuses on high-level features such as lane markings, and pavement for immediate decision making. Thus, our processed rendered images still contain relevant information for the model to predict correct control commands.…”

Section: B Data Augmentationmentioning

confidence: 99%

Robust Autonomous Vehicle Pursuit Without Expert Steering Labels

Pan,

Zhou,

Gladkova

et al. 2023

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

In this work, we present a learning method for both lateral and longitudinal motion control of an ego-vehicle for the task of vehicle pursuit. The car being controlled does not have a pre-defined route, rather it reactively adapts to follow a target vehicle while maintaining a safety distance. To train our model, we do not rely on steering labels recorded from an expert driver, but effectively leverage a classical controller as an offline label generation tool. In addition, we account for the errors in the predicted control values, which can lead to a loss of tracking and catastrophic crashes of the controlled vehicle. To this end, we propose an effective data augmentation approach, which allows to train a network that is capable of handling different views of the target vehicle. During the pursuit, the target vehicle is firstly localized using a Convolutional Neural Network. The network takes a single RGB image along with cars' velocities and estimates target vehicle's pose with respect to the ego-vehicle. This information is then fed to a Multi-Layer Perceptron, which regresses the control commands for the ego-vehicle, namely throttle and steering angle. We extensively validate our approach using the CARLA simulator on a wide range of terrains. Our method demonstrates realtime performance, robustness to different scenarios including unseen trajectories and high route completion. Project page containing code and multimedia can be publicly accessed here: https://changyaozhou.github.io/Autonomous-Vehicle-Pursuit/.

show abstract

Towards Generalizing Sensorimotor Control Across Weather Conditions

Cited by 7 publications

References 30 publications

Lateral Ego-Vehicle Control without Supervision using Point Clouds

Lateral Ego-Vehicle Control without Supervision using Point Clouds

Multi-Vehicle Trajectory Prediction at Intersections using State and Intention Information

Robust Autonomous Vehicle Pursuit Without Expert Steering Labels

Contact Info

Product

Resources

About