The control theory of motion-based communication: Problems in teaching robots to dance

Abstract

The paper describes results on two components of a research program focused on motion-based communication mediated by the dynamics of a control system. Specifically we are interested in how mobile agents engaged in a shared activity such as dance can use motion as a medium for transmitting certain types of messages. The first part of the paper adopts the terminology of motion description languages and deconstructs an elementary form of the well-known popular dance, Salsa, in terms of four motion primitives (dance steps). Several notions of dance complexity are introduced. We describe an experiment in which ten performances by an actual pair of dancers are evaluated by judges and then compared in terms of proposed complexity metrics. An energy metric is also defined. Values of this metric are obtained by summing the lengths of motion segments executed by wheeled robots replicating the movements of the human dancers in each of the ten dance performances. Of all the metrics that are considered in this experiment, energy is the most closely correlated with the human judges' assessments of performance quality.The second part of the paper poses a general class of dual objective motion control problems in which a primary objective (artistic execution of a dance step or efficient movement toward a specified terminal state) is combined with a communication objective. Solutions of varying degrees of explicitness can be given in several classes of problems of communicating through the dynamics of finite dimensional linear control systems. In this setting it is shown that the cost of adding a communication component to motions that steer a system between prescribed pairs of states is independent of those states. At the same time, the optimal encoding problem itself is shown to be a problem of packing geometric objects,and it remains open. Current research is aimed at solving such communication-through-action problems in the context of the motion control of mobile robots.John Baillieul is with the Boston University,

Information-Based Multi-Agent Systems

“…Recent research has been aimed at developing a theoretical foundation of actionmediated communication. Communication protocols for motion-based signaling between mobile robots have been developed (Raghunathan and Baillieul 2009) and preliminary steps towards a theory of artistic expression through controlled movements in dance have been reported in Baillieul and Özcimder (2012). Motion-based communication of this type involves specially tailored motion description languages in which sequences of motion primitives are assembled with the objective of conveying artistic intent, while minimizing the use of limited energy resources in carrying out the movement.…”

Communication through motion in dance with topological constraints

“…10. In [2], Beginner Level Performance is introduced as a set of four distinct Salsa moves {A, B, C, D}. In [3] this is extended to Intermediate Level Salsa (ILS) with eleven moves.…”

“…In our recent work [2], we propose a mathematical model by adapting the terminology of motion description languages in order to represent a salsa performance. This allows us to represent a dance performance by concatenation of letters that are assigned for each move achieved in eight beat musical intervals.…”

“…This allows us to represent a dance performance by concatenation of letters that are assigned for each move achieved in eight beat musical intervals. In [2], the set of four fundamental moves in salsa is defined as the Beginner Salsa Performance (BSP). We proposed complexity metrics based on the energy consumption and the sequence entropy as a metric in order to measure the letter diversity in a dance sequence.…”

Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation

“…Other authors have pursued applications of control algorithms to art [3], [15], [19], [20], [21], [32], and the emerging applications signal a growing maturity of control technology in its ability to coexist with people.…”