Learning Multi-Object Tracking and Segmentation from Automatic Annotations

Porzi, Lorenzo; Hofinger, Markus; Ruiz, Idoia; Serrat, Joan; Bulò, Samuel Rota; Kontschieder, Peter

doi:10.48550/arxiv.1912.02096

Cited by 4 publications

(11 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, to test the generalization ability of instance association, we test PointTrack*, whose instance embeddings extraction is only fine-tuned on KITTI MOTS, on both MOTSChallenge and Apollo MOTS. We compare recent works on MOTS: TRCNN [35], MOTSNet [28], BePix [32], and MOTSFusion (online) [18]. TRCNN and MOTSNet perform 2D tracking while BePix and MOTSFusion track on 3D.…”

Section: Methodsmentioning

confidence: 99%

“…Unlike 2D bounding boxes which might overlap heavily in crowded scenes, per-pixel segments locate objects precisely. Recently instance segments have been exploited for improving the tracking performance [19,27,26,12,28]. In [26], Osep et al present a model-free multi-object tracking approach that uses a category-agnostic image segmentation method to track objects.…”

Section: Related Workmentioning

confidence: 99%

“…Track-RCNN [35] extends Mask-RCNN with 3D convolutions to incorporate temporal information and extracts instance embeddings for tracking by ROI Align. MOTSNet [28] proposes a mask pooling layer to Mask-RCNN to improve object association over time. STE [12] introduces a new spatial-temporal embedding loss to generate temporally consistent instance segmentation and regard the mean embeddings of all pixels on segments as the instance embedding for data association.…”

Section: Related Workmentioning

confidence: 99%

“…Different from previous methods [35,28] which put great efforts to adapt mask RCNN into the MOTS frameworks, PointTrack builds on a state-of-the-art onestage instance segmentation method named SpatialEmbedding [24]. SpatialEmbedding performs instance segmentation without bbox proposals, and thus runs much faster than two-stage methods.…”

Section: Instance Segmentation With Temporal Seed Consistencymentioning

confidence: 99%

“…TRCNN [35] extends Mask RCNN by 3D convolutions and adopts ROI Align to extract instance embeddings in bbox proposals. To focus on the segment area in feature extraction, Porzi et al [28] propose mask pooling to replace ROI Align. Nevertheless, as affected by the receptive field of convolutions, the foreground features and the background features are still mixed up, which is harmful for learning discriminative feature.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Segment as Points for Efficient Online Multi-Object Tracking and Segmentation

Zhang

Tan

et al. 2020

Preprint

View full text Add to dashboard Cite

Current multi-object tracking and segmentation (MOTS) methods follow the tracking-by-detection paradigm and adopt convolutions for feature extraction. However, as affected by the inherent receptive field, convolution based feature extraction inevitably mixes up the foreground features and the background features, resulting in ambiguities in the subsequent instance association. In this paper, we propose a highly effective method for learning instance embeddings based on segments by converting the compact image representation to un-ordered 2D point cloud representation. Our method generates a new tracking-by-points paradigm where discriminative instance embeddings are learned from randomly selected points rather than images. Furthermore, multiple informative data modalities are converted into point-wise representations to enrich point-wise features. The resulting online MOTS framework, named PointTrack, surpasses all the state-of-the-art methods including 3D tracking methods by large margins (5.4% higher MOTSA and 18 times faster over MOTSFusion) with the near real-time speed (22 FPS). Evaluations across three datasets demonstrate both the effectiveness and efficiency of our method. Moreover, based on the observation that current MOTS datasets lack crowded scenes, we build a more challenging MOTS dataset named APOLLO MOTS with higher instance density. Both APOLLO MOTS and our codes are publicly available at https://github.com/detectRecog/PointTrack.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Instance Segmentation With Temporal Seed Consistencymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Segment as Points for Efficient Online Multi-Object Tracking and Segmentation

Zhang

Tan

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Fast forward approximation and multitask inversion of gravity anomaly based on UNet3+

Zhang

2023

Geophysical Journal International

View full text Add to dashboard Cite

Summary Gravity inversion is a process that obtains the spatial structure and physical properties of underground anomalies using surface collected gravity anomaly data. In recent years, the rapid development of deep learning (DL) has enabled the achievement of good results for gravity inversion methods based on DL. These methods aim to learn the mapping between geological models and gravity anomaly data by training a neural network with geological models as labels. However, using DL inversion requires generating a large amount of training data for each geological target and involves the forward calculation of the generated models, which inevitably consumes a large amount of time and storage space. To address this issue, we propose using a neural network to approximate the expensive forward computation with a fast evaluation alternative. After training, the network can reproduce gravity anomalies at any observation point. To evaluate the effectiveness of the forward model, we use the gravity anomalies predicted by the forward network for inversion network training. Additionally, to mitigate the problem of poor generalization of existing DL inversions, we propose using multi-task learning (MTL). By learning multiple related tasks simultaneously, the generalization ability of the model improves, thus enhancing the performance of the main task. In this paper, a multi-task UNet3+ network is proposed to realize anomaly bodies localization and density contrasts reconstruction simultaneously. Test results on the synthetic dataset demonstrate that the gravity anomalies predicted by the forward network can be successfully inverted, and the multi-task approach can predict subsurface geology more accurately than the single-task. To further illustrate the effectiveness of the algorithm, we apply this method to the inversion of the San Nicolas deposit in central Mexico.

show abstract

Enhancing Image Annotation With Object Tracking and Image Retrieval: A Systematic Review

Fernandes,

Pessoa,

Salgado

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Effective image and video annotation is a fundamental pillar in computer vision and artificial intelligence, crucial for the development of accurate machine learning models. Object tracking and image retrieval techniques are essential in this process, significantly improving the efficiency and accuracy of automatic annotation. This paper systematically investigates object tracking and image acquisition techniques. It explores how these technologies can collectively enhance the efficiency and accuracy of the annotation processes for image and video datasets. Object tracking is examined for its role in automating annotations by tracking objects across video sequences, while image retrieval is evaluated for its ability to suggest annotations for new images based on existing data. The review encompasses diverse methodologies, including advanced neural networks and machine learning techniques, highlighting their effectiveness in various contexts like medical analyses and urban monitoring. Despite notable advancements, challenges such as algorithm robustness and effective human-AI collaboration are identified. This review provides valuable insights into these technologies' current state and future potential in improving image annotation processes, even showing existing applications of these techniques and their full potential when combined.

show abstract

Learning Multi-Object Tracking and Segmentation from Automatic Annotations

Abstract: es * Figure 1: KITTI sub-sequences with automatically generated MOTS annotations as color-coded instance masks (left to right).

Cited by 4 publications

References 43 publications

Segment as Points for Efficient Online Multi-Object Tracking and Segmentation

Segment as Points for Efficient Online Multi-Object Tracking and Segmentation

Fast forward approximation and multitask inversion of gravity anomaly based on UNet3+

Enhancing Image Annotation With Object Tracking and Image Retrieval: A Systematic Review

Contact Info

Product

Resources

About