Understanding and Exploiting Object Interaction Landscapes

Abstract

Figure 1: Different interaction landscapes representing the interactions of a motion driver with a static object. We capture the motion trajectories (red) and encode their signatures into a descriptor that can be used for comparing interactions. From left to right: a cloth simulation interacting with a support structure, a human model walking on a floor, a wind simulation interacting with a car, and a robotic hand grasping a cup. AbstractInteractions play a key role in understanding objects and scenes, for both virtual and real world agents. We introduce a new general representation for proximal interactions among physical objects that is agnostic to the type of objects or interaction involved. The representation is based on tracking particles on one of the participating objects and then observing them with sensors appropriately placed in the interaction volume or on the interaction surfaces. We show how to factorize these interaction descriptors and project them into a particular participating object so as to obtain a new functional descriptor for that object, its interaction landscape, capturing its observed use in a spatio-temporal framework. Interaction landscapes are independent of the particular interaction and capture subtle dynamic effects in how objects move and behave when in functional use. Our method relates objects based on their function, establishes correspondences between shapes based on functional key points and regions, and retrieves peer and partner objects with respect to an interaction.

Aura Mesh: Motion Retargeting to Preserve the Spatial Relationships between Skinned Characters

“…We use every skin vertex as candidate interaction points to express the spatial relationships, indicating the desired positions of the corresponding skin vertices of the targeted character. Our method can be improved with respect to selecting the candidate interaction points by using a distribution of sample points that can emphasize the characteristics of the surface [PKH*17]. In addition to the interaction points, additional types of interaction indicators may enhance the capability of capturing and retargeting interaction semantics.…”

Functionality Representations and Applications for Shape Analysis

“…The Interaction Landscapes descriptor [PKH*17] is the first attempt along this direction of describing atomic interactions. To simplify the acquisition and modeling of the interactions, only one interacting entity is considered.…”

“…Components of the interaction landscapes representation of Pirk et al [PKH*17]. Examples of sensor regions in the domain around the observed shape, shown in (a), and motion particles on the surface of the moving model with direction and speed of movement recorded, shown in (b).…”

“…Note that the boundary between the entities in these works is in fact a portion of space between the entities, which can be empty when the entities are far apart, or which can possess an infinitesimal thickness when the entities are touching. Dense volume feature (VF): represents the space containing the two entities, capturing scalar or vector features of the interaction at a dense set of points in the containing volume. An example of this representation are Interaction Landscapes [PKH*17]. Gestalt and symmetry grouping (SG): includes relations that capture translational and rotational symmetry groupings of the entities, and repeated geometric patterns formed by the entities.…”

“… Dynamic field: the time‐varying nature of a given atomic interaction is captured in a volume containing the interacting entities. For example, the Interaction Landscapes descriptor [PKH*17] represents the dynamic evolution of a relative position interaction as a scalar or vector field over the volume. Note that, as a special case, one of the interacting entities can be a human agent.…”