Data association for semantic world modeling from partial views

Wong, Lawson L. S.; Kaelbling, Leslie Pack; Tom,; Lozano-P$#, s; rez,

doi:10.1177/0278364914559754

Cited by 29 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the occlusions are handled by means of a multiple hypothesis tracker, which is suited for short occlusions rather then long occlusions. The limitations with the use of multiple hypothesis tracking for world modeling, and consequently also for handling object occlusions in anchoring scenarios (as in Elfring et al, 2013), have likewise been pointed out in a publication by Wong et al (2015). Wong et al reported instead the use of a clustering-based data association approach (opposed to a tracking-based approach), in order to aggregate a consistent semantic world model from multiple viewpoints, and hence, compensate for partial occlusions from a single viewpoint perspective of the scene.…”

Section: Occlusionsmentioning

confidence: 86%

Symbolic Learning and Reasoning With Noisy Data for Probabilistic Anchoring

et al. 2020

View full text Add to dashboard Cite

Robotic agents should be able to learn from sub-symbolic sensor data and, at the same time, be able to reason about objects and communicate with humans on a symbolic level. This raises the question of how to overcome the gap between symbolic and sub-symbolic artificial intelligence. We propose a semantic world modeling approach based on bottom-up object anchoring using an object-centered representation of the world. Perceptual anchoring processes continuous perceptual sensor data and maintains a correspondence to a symbolic representation. We extend the definitions of anchoring to handle multi-modal probability distributions and we couple the resulting symbol anchoring system to a probabilistic logic reasoner for performing inference. Furthermore, we use statistical relational learning to enable the anchoring framework to learn symbolic knowledge in the form of a set of probabilistic logic rules of the world from noisy and sub-symbolic sensor input. The resulting framework, which combines perceptual anchoring and statistical relational learning, is able to maintain a semantic world model of all the objects that have been perceived over time, while still exploiting the expressiveness of logical rules to reason about the state of objects which are not directly observed through sensory input data. To validate our approach we demonstrate, on the one hand, the ability of our system to perform probabilistic reasoning over multi-modal probability distributions, and on the other hand, the learning of probabilistic logical rules from anchored objects produced by perceptual observations. The learned logical rules are, subsequently, used to assess our proposed probabilistic anchoring procedure. We demonstrate our system in a setting involving object interactions where object occlusions arise and where probabilistic inference is needed to correctly anchor objects.

show abstract

Section: Occlusionsmentioning

confidence: 86%

Symbolic Learning and Reasoning With Noisy Data for Probabilistic Anchoring

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Two canonical algorithms are Joint Probability Data Association (JPDA) [34] and Multiple Hypothesis Tracking (MHT) [35]. These techniques have been applied to many problems including human following [36], object tracking [37] and human-robot interaction [38]. However, JPDA requires solving the data association problem which is especially costly when the actual number of targets is not known exactly [12].…”

Section: Related Workmentioning

confidence: 99%

GM-PHD Filter for Searching and Tracking an Unknown Number of Targets with a Mobile Sensor with Limited FOV

Sung¹,

Tokekar²

2022

Preprint

View full text Add to dashboard Cite

We study the problem of searching for and tracking a collection of moving targets using a robot with a limited Field-Of-View (FOV) sensor. The actual number of targets present in the environment is not known a priori. We propose a search and tracking framework based on the concept of Bayesian Random Finite Sets (RFSs). Specifically, we generalize the Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter which was previously applied for tracking problems to allow for simultaneous search and tracking with a limited FOV sensor. The proposed framework can extract individual target tracks as well as estimate the number and the spatial density of targets. We also show how to use the Gaussian Process (GP) regression to extract and predict unknown target trajectories in this framework. We demonstrate the efficacy of our techniques through representative simulations and a real data collected from an aerial robot.

show abstract

“…Several authors have shown how a world model based on objects with attributes that are updated over time can be used to deal with the challenges of complex and occluded environments (Elfring et al, 2013;Persson et al, 2020;Wong et al, 2015a). These approaches have in common the use of multiobject tracking (MOT) to link upcoming noisy and uncertain measurements from the sensors of the robot with the representation of the objects that generated those measurements.…”

Section: Introductionmentioning

confidence: 99%

Development and evaluation of automated localization and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking

Rincon¹,

Henten²,

Kootstra³

2022

Preprint

View full text Add to dashboard Cite

Accurate representation and localization of relevant objects is important for robots to perform tasks. Building a generic representation that can be used across different environments and tasks is not easy, as the relevant objects vary depending on the environment and the task. Furthermore, another challenge arises in agro-food environments due to their complexity, and high levels of clutter and occlusions. In this paper, we present a method to build generic representations in highly occluded agro-food environments using multi-view perception and 3D multi-object tracking. Our representation is built upon a detection algorithm that generates a partial point cloud for each detected object. The detected objects are then passed to a 3D multi-object tracking algorithm that creates and updates the representation over time. The whole process is performed at a rate of 10 Hz. We evaluated the accuracy of the representation on a real-world agro-food environment, where it was able to successfully represent and locate tomatoes in tomato plants despite a high level of occlusion. We were able to estimate the total count of tomatoes with a maximum error of 5.08% and to track tomatoes with a tracking accuracy up to 71.47%. Additionally, we showed that an evaluation using tracking metrics gives more insight in the errors in localizing and representing the fruits.

show abstract

Data association for semantic world modeling from partial views

Cited by 29 publications

References 36 publications

Symbolic Learning and Reasoning With Noisy Data for Probabilistic Anchoring

Symbolic Learning and Reasoning With Noisy Data for Probabilistic Anchoring

GM-PHD Filter for Searching and Tracking an Unknown Number of Targets with a Mobile Sensor with Limited FOV

Development and evaluation of automated localization and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking

Contact Info

Product

Resources

About