LCrowdV: Generating Labeled Videos for Simulation-Based Crowd Behavior Learning

Cheung, Ernest; Wong, Tsan Kwong; Bera, Aniket; Wang, Xiaogang; Manocha, Dinesh

doi:10.1007/978-3-319-48881-3_50

Cited by 21 publications

(17 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semi-synthetic approaches can employ straightforward "cut and paste" methods like [3,9,13] or more sophisticated techniques which focus on the augmentation of the static real scenes with CAD objects such as cars [1] or pedestrians [7].…”

Section: Related Workmentioning

confidence: 99%

Attention-based Adversarial Appearance Learning of Augmented Pedestrians

Strauss,

Savkin,

Tombari

2021

Preprint

View full text Add to dashboard Cite

Synthetic data became already an essential component of machine learning-based perception in the field of autonomous driving. Yet it still cannot replace real data completely due to the sim2real domain shift. In this work, we propose a method that leverages the advantages of the augmentation process and adversarial training to synthesize realistic data for the pedestrian recognition task. Our approach utilizes an attention mechanism driven by an adversarial loss to learn domain discrepancies and improve sim2real adaptation. Our experiments confirm that the proposed adaptation method is robust to such discrepancies and reveals both visual realism and semantic consistency. Furthermore, we evaluate our data generation pipeline on the task of pedestrian recognition and demonstrate that generated data resemble properties of the real domain.

show abstract

Section: Related Workmentioning

confidence: 99%

Attention-based Adversarial Appearance Learning of Augmented Pedestrians

Strauss,

Savkin,

Tombari

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Their analysis for pedestrians observed by cameras in a commercial street shows that up to 70% of people are moving in groups, including couples, families, or friends, and the group sizes follow a Poisson distribution. Since video-based crowd behavior learning needs labeled video datasets, the study in [5] aims to generate synthetic labels and combine them with real videos. While Group-In also has a clustering-based approach, it does not require camera deployment or training using labeled video datasets.…”

Section: Related Workmentioning

confidence: 99%

Group-In: Group Inference from Wireless Traces of Mobile Devices

Solmaz¹,

Fürst²,

Aytaç³

et al. 2020

2020 19th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)

View full text Add to dashboard Cite

This paper proposes Group-In, a wireless scanning system to detect static or mobile people groups in indoor or outdoor environments. Group-In collects only wireless traces from the Bluetooth-enabled mobile devices for group inference. The key problem addressed in this work is to detect not only static groups but also moving groups with a multi-phased approach based only noisy wireless Received Signal Strength Indicator (RSSIs) observed by multiple wireless scanners without localization support. We propose new centralized and decentralized schemes to process the sparse and noisy wireless data, and leverage graph-based clustering techniques for group detection from short-term and long-term aspects. Group-In provides two outcomes: 1) group detection in short time intervals such as two minutes and 2) long-term linkages such as a month. To verify the performance, we conduct two experimental studies. One consists of 27 controlled scenarios in the lab environments. The other is a real-world scenario where we place Bluetooth scanners in an office environment, and employees carry beacons for more than one month. Both the controlled and real-world experiments result in high accuracy group detection in short time intervals and sampling liberties in terms of the Jaccard index and pairwise similarity coefficient. KEYWORDS ubiquitous and mobile computing, group detection, human mobility, wireless, internet of things (a) Finding groups in urban areas.

show abstract

“…Although the concept of generating training datasets using simulators for CV is quite new, there are some recent precedents in both microscopy and other GT hard problems. Datasets for tracking unmanned aerial vehicles [27], climate [28], object interaction and physical event prediction [29,30], image understanding [31], and crowd behaviour [32] were synthesized using physics or other behaviour emulating approaches. Within microscopy, we are aware of a microscopy image simulator [33], synthetic dataset for cells 1 , as well as a dataset for sub-cellular spherical structures called vesicles [34].…”

Section: Related Workmentioning

confidence: 99%

Simulation-supervised deep learning for analysing organelles states and behaviour in living cells

Sekh¹,

Opstad²,

Agarwal³

et al. 2020

Preprint

View full text Add to dashboard Cite

In many real-world scientific problems, generating ground truth (GT) for supervised learning is almost impossible. The causes include limitations imposed by scientific instrument, physical phenomenon itself, or the complexity of modeling. Performing artificial intelligence (AI) tasks such as segmentation, tracking, and analytics of small sub-cellular structures such as mitochondria in microscopy videos of living cells is a prime example. The 3D blurring function of microscope, digital resolution from pixel size, optical resolution due to the character of light, noise characteristics, and complex 3D deformable shapes of mitochondria, all contribute to making this problem GT hard. Manual segmentation of 100s of mitochondria across 1000s of frames and then across many such videos is not only herculean but also physically inaccurate because of the instrument and phenomena imposed limitations. Unsupervised learning produces less than optimal results and accuracy is important if inferences relevant to therapy are to be derived. In order to solve this unsurmountable problem, we bring modeling and deep learning to a nexus. We show that accurate physics based modeling of microscopy data including all its limitations can be the solution for generating simulated training datasets for supervised learning. We show here that our simulation-supervised segmentation approach is a great enabler for studying mitochondrial states and behaviour in heart muscle cells, where mitochondria have a significant role to play in the health of the cells. We report unprecedented mean IoU score of 91% for binary segmentation (19% better than the best performing unsupervised approach) of mitochondria in actual microscopy videos of living cells. We further demonstrate the possibility of performing multi-class classification, tracking, and morphology associated analytics at the scale of individual mitochondrion.

show abstract

LCrowdV: Generating Labeled Videos for Simulation-Based Crowd Behavior Learning

Cited by 21 publications

References 69 publications

Attention-based Adversarial Appearance Learning of Augmented Pedestrians

Attention-based Adversarial Appearance Learning of Augmented Pedestrians

Group-In: Group Inference from Wireless Traces of Mobile Devices

Simulation-supervised deep learning for analysing organelles states and behaviour in living cells

Contact Info

Product

Resources

About