Semantic analysis of manipulation actions using spatial relations

Ziaeetabar, Fatemeh; Aksoy, Eren Erdal; Wörgötter, Florentin; Tamošiūnaitė, Minija

doi:10.1109/icra.2017.7989536

Cited by 20 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If a point is in AABB, the next step is to determine whether the point is in the convex hull. AABB is employed to represent the boundary of objects [ 16 ]. However, it is inappropriate for the spatial topological relation analysis.…”

Section: Methodsmentioning

confidence: 99%

“…Spatial topological relations describe the adjacency and association relations about spatial points, lines, and surfaces [ 15 ]. A correct understanding of the spatial topological relations between objects is essential for the successful execution of robot actions [ 16 ]. The behavior decision of robots depends on the current state of the spatial topological relations [ 17 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spatial Topological Relation Analysis for Cluttered Scenes

Zhang

et al. 2020

Sensors

View full text Add to dashboard Cite

The spatial topological relations are the foundation of robot operation planning under unstructured and cluttered scenes. Defining complex relations and dealing with incomplete point clouds from the surface of objects are the most difficult challenge in the spatial topological relation analysis. In this paper, we presented the classification of spatial topological relations by dividing the intersection space into six parts. In order to improve accuracy and reduce computing time, convex hulls are utilized to represent the boundary of objects and the spatial topological relations can be determined by the category of points in point clouds. We verified our method on the datasets. The result demonstrated that we have great improvement comparing with the previous method.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Spatial Topological Relation Analysis for Cluttered Scenes

Zhang

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Describing action with a grammatical structure [25][26][27] such as eSEC [22,23], renders a simple and fast framework for recognition and prediction in the presence of unknown objects and noise. This robustness lends itself to an intriguing hypothesis, which is asking to what degree such an event-based framework might help young infants to bootstrap action knowledge in view of the vast number of objects that they have never encountered before.…”

Section: Plos Onementioning

confidence: 99%

“…As the basis for spatial relation calculation we use extended semantic event chains (eSEC), introduced in our previous work for action recognition in computer vision [22]. This approach allows us to determine a sequence of discrete spatial relations between different objects in the scene throughout the manipulation.…”

Section: Introductionmentioning

confidence: 99%

Using enriched semantic event chains to model human action prediction based on (minimal) spatial information

et al. 2020

Self Cite

View full text Add to dashboard Cite

Predicting other people’s upcoming action is key to successful social interactions. Previous studies have started to disentangle the various sources of information that action observers exploit, including objects, movements, contextual cues and features regarding the acting person’s identity. We here focus on the role of static and dynamic inter-object spatial relations that change during an action. We designed a virtual reality setup and tested recognition speed for ten different manipulation actions. Importantly, all objects had been abstracted by emulating them with cubes such that participants could not infer an action using object information. Instead, participants had to rely only on the limited information that comes from the changes in the spatial relations between the cubes. In spite of these constraints, participants were able to predict actions in, on average, less than 64% of the action’s duration. Furthermore, we employed a computational model, the so-called enriched Semantic Event Chain (eSEC), which incorporates the information of different types of spatial relations: (a) objects’ touching/untouching, (b) static spatial relations between objects and (c) dynamic spatial relations between objects during an action. Assuming the eSEC as an underlying model, we show, using information theoretical analysis, that humans mostly rely on a mixed-cue strategy when predicting actions. Machine-based action prediction is able to produce faster decisions based on individual cues. We argue that human strategy, though slower, may be particularly beneficial for prediction of natural and more complex actions with more variable or partial sources of information. Our findings contribute to the understanding of how individuals afford inferring observed actions’ goals even before full goal accomplishment, and may open new avenues for building robots for conflict-free human-robot cooperation.

show abstract

“…The sub-tables below show how these relations change in the course of an action. This tabular action representation is called: Extended Semantic Event Chain (ESEC [9]).…”

Section: Actionsmentioning

confidence: 99%

Helping a robot to understand human actions and objects: a grammatical view

Wörgötter

Tamošiūnaitė

2020

Artif Life Robotics

Self Cite

View full text Add to dashboard Cite

Humans are able to perform a wide variety of complex actions manipulating a very large number of objects. We can make predictions on the outcome of our actions and on how to use different objects. Hence, we have excellent action and object understanding. Artificial agents, on the other hand, still miserably fail in this respect. It is particularly puzzling how inexperienced, young humans can acquire such knowledge; bootstrapped by exploration and extended by supervision. In this study we have, therefore, addressed the question how to structure the realm of actions and objects into dynamic representations, which allow for the easy learning of different action and object concepts. Performing different manipulation actions on a table top (e.g. the actions of “making a breakfast”), we show with our robots that this will indeed lead to some kind of implicit (un-reflected) understanding of action and object concepts allowing the agent to generalize actions and redefine object uses according to need.

show abstract

Semantic analysis of manipulation actions using spatial relations

Cited by 20 publications

References 20 publications

Spatial Topological Relation Analysis for Cluttered Scenes

Spatial Topological Relation Analysis for Cluttered Scenes

Using enriched semantic event chains to model human action prediction based on (minimal) spatial information

Helping a robot to understand human actions and objects: a grammatical view

Contact Info

Product

Resources

About