Road Layout Understanding by Generative Adversarial Inpainting

Berlincioni, Lorenzo; Becattini, Federico; Galteri, Leonardo; Seidenari, Lorenzo; Bimbo, Alberto Del

doi:10.1007/978-3-030-25614-2_10

Cited by 14 publications

(14 citation statements)

References 43 publications

(76 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interaction is still a matter of primary importance but is now considered as the analysis of social patterns that determine how groups move together in a shared space [6], [7], [40]. At the same time, for urban navigation, agents need to infer the layout of the scene, recovering possible occlusions that they might have from their point of view [41], [42]. A correct understanding of both context and other agents is pivotal for safety, since it enables the anticipation of dangerous situations such as car accidents [43].…”

Section: Related Workmentioning

confidence: 99%

Multiple Trajectory Prediction of Moving Agents With Memory Augmented Networks

Marchetti

Becattini

Seidenari

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Pedestrians and drivers are expected to safely navigate complex urban environments along with several non cooperating agents. Autonomous vehicles will soon replicate this capability. Each agent acquires a representation of the world from an egocentric perspective and must make decisions ensuring safety for itself and others. This requires to predict motion patterns of observed agents for a far enough future. In this paper we propose MANTRA, a model that exploits memory augmented networks to effectively predict multiple trajectories of other agents, observed from an egocentric perspective. Our model stores observations in memory and uses trained controllers to write meaningful pattern encodings and read trajectories that are most likely to occur in future. We show that our method is able to natively perform multi-modal trajectory prediction obtaining state-of-the art results on four datasets. Moreover, thanks to the non-parametric nature of the memory module, we show how once trained our system can continuously improve by ingesting novel patterns.

show abstract

Section: Related Workmentioning

confidence: 99%

Multiple Trajectory Prediction of Moving Agents With Memory Augmented Networks

Marchetti

Becattini

Seidenari

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Lee et al [13] propose a network that predicts the location and the shape of different objects (namely pedestrian and car) to insert them in the semantic segmentation space. Finally, Berlincioni et al [28] focus on semantic segmentation removing cars and pedestrians from a road layout by feeding the binary masks in input to the model. However, existing work allow users to either insert or delete objects from an existing scene.…”

Section: Related Workmentioning

confidence: 99%

Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation

Ardino¹,

Liu²,

Ricci³

et al. 2020

Preprint

View full text Add to dashboard Cite

Manipulating images of complex scenes to reconstruct, insert and/or remove specific object instances is a challenging task. Complex scenes contain multiple semantics and objects, which are frequently cluttered or ambiguous, thus hampering the performance of inpainting models. Conventional techniques often rely on structural information such as object contours in multi-stage approaches that generate unreliable results and boundaries. In this work, we propose a novel deep learning model to alter a complex urban scene by removing a userspecified portion of the image and coherently inserting a new object (e.g. car or pedestrian) in that scene. Inspired by recent works on image inpainting, our proposed method leverages the semantic segmentation to model the content and structure of the image, and learn the best shape and location of the object to insert. To generate reliable results, we design a new decoder block that combines the semantic segmentation and generation task to guide better the generation of new objects and scenes, which have to be semantically consistent with the image. Our experiments, conducted on two large-scale datasets of urban scenes (Cityscapes and Indian Driving), show that our proposed approach successfully address the problem of semantically-guided inpainting of complex urban scene.

show abstract

“…In addition, data acquired from sensors might not have access to all desired informations, which could instead be acquired in a synthetic or simulated environment. An example of this is occlusion caused by other vehicles, which has been addressed using GANs to generate samples recovering the structure of the layout [26], [27]. In the case of trajectory data, what can be observed in the real world is only the path taken by a vehicle.…”

Section: Introductionmentioning

confidence: 99%

Multiple Future Prediction Leveraging Synthetic Trajectories

Berlincioni

Becattini

Seidenari

et al. 2021

2020 25th International Conference on Pattern Recognition (ICPR)

Self Cite

View full text Add to dashboard Cite

Trajectory prediction is an important task, especially in autonomous driving. The ability to forecast the position of other moving agents can yield to an effective planning, ensuring safety for the autonomous vehicle as well for the observed entities. In this work we propose a data driven approach based on Markov Chains to generate synthetic trajectories, which are useful for training a multiple future trajectory predictor. The advantages are twofold: on the one hand synthetic samples can be used to augment existing datasets and train more effective predictors; on the other hand, it allows to generate samples with multiple ground truths, corresponding to diverse equally likely outcomes of the observed trajectory. We define a trajectory prediction model and a loss that explicitly address the multimodality of the problem and we show that combining synthetic and real data leads to prediction improvements, obtaining state of the art results.

show abstract

Road Layout Understanding by Generative Adversarial Inpainting

Cited by 14 publications

References 43 publications

Multiple Trajectory Prediction of Moving Agents With Memory Augmented Networks

Multiple Trajectory Prediction of Moving Agents With Memory Augmented Networks

Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation

Multiple Future Prediction Leveraging Synthetic Trajectories

Contact Info

Product

Resources

About