Sequential Gesture Learning for Continuous Labanotation Generation Based on the Fusion of Graph Neural Networks

Long and short spatial-temporal relations model for automatic generation of Labanotation

“…Connectionist temporal classification (CTC) 18 based Labanotation transcription uses directed acyclic graph (DAG) to gain temporal dependency and spatial dependency separately.…”

“…At present, research on automatic generation of Labanotation, the one-stage model introduces the Li Group Characteristics to describe the spatial relationship between the human skeleton joint points, 16,17,20,21 and the two-stage model introduces the directed graph to model the spatial and temporal correlation between the human skeleton joint point sequences. 18 All methods use manual construction of the relationship among human skeleton joint points, which may not be the best description of the relationship among human skeleton joint points. To solve this problem, the gesture-sensitive graph convolutional network (GS-GCN) model was proposed and applied to learning relationship among joint points of human natural skeleton.…”

“…22 However, the method ignores the important role of non-adjacent skeleton sequences and cannot extract the characteristics of complex dance movements. In addition, most of the research on automatic generation of Labanotation, in terms of skeleton sequences representation, the early research only considered the spatial dependence between modeling skeleton sequences 18 or the temporal dependence. [16][17][18][19][20] These methods did not consider the relationship between time and space of skeleton sequences.…”

“…At present, many methods realize this task such as the traditional computer-based methods 1 – 8 improve the efficiency of Labanotation generation, but this method still needs the guidance of experts. Later deep learning methods 9 – 22 automatically generate Labanotation, which solves the problem of time-consuming and laborious generation of Labanotation based on a computer. These methods are divided into two-stage methods and one-stage methods.…”

“…In addition, most of the research on automatic generation of Labanotation, in terms of skeleton sequences representation, the early research only considered the spatial dependence between modeling skeleton sequences 18 or the temporal dependence 16 – 20 These methods did not consider the relationship between time and space of skeleton sequences. For example, the rotation movement start and end of dance movements are strongly correlated, but separately modeling them may not be able to predict dance movements from capturing bad relationship information.…”