Finger-String Interaction Analysis in Guitar Playing With Optical Motion Capture

Abstract

We present a method for the analysis of the finger-string interaction in guitar performances and the computation of fine actions during the plucking gesture. The method is based on Motion Capture using high-speed cameras that can track the position of reflective markers placed on the guitar and fingers, in combination with audio analysis. A major problem inherent in optical motion capture is that of marker occlusion and, in guitar playing, it is the right hand of the guitarist that is extremely difficult to capture, especially during the plucking process, where the track of the markers at the fingertips is lost very frequently. This work presents two models that allow the reconstruction of the position of occluded markers: a rigid-body model to track the motion of the guitar strings and a flexible-body model to track the motion of the hands. In combination with audio analysis (onset and pitch detection), the method can estimate a comprehensive set of sound control features that include the plucked string, the plucking finger, and the characteristics of the plucking gesture in the phases of contact, pressure and release (e.g., position, timing, velocity, direction, or string displacement).

Correlations between body postures and musculoskeletal pain in guitar players

“…The subject was situated in a room, equipped with 6 motion capture cameras (Qualysis, Sweden). This method for quantifying upper limb kinematics was found to be reliable and provide good accuracy for tasks such as keyboard typing [ 22 ] and guitar playing [ 23 ]. After calibration of the system and recording of a static trial, each subject played a familiar 2-minute tune, suitable for the acoustic guitar, provided for this study and data were captured at 100Hz.…”

Evaluation of Inertial Sensor Data by a Comparison with Optical Motion Capture Data of Guitar Strumming Gestures

“…Works dedicated to guitarists’ gestures are not very common, especially those that focus on the strumming technique. They can be grouped into: creative applications in real-time [ 26 , 27 ]; development of new interfaces or augmented instruments [ 28 , 29 , 30 , 31 ]; and, performance analysis [ 32 , 33 , 34 , 35 ].…”

“…Freire et al [ 33 ] describes a study of microtiming features present in accompaniment patterns played with strummed chords. Perez-Carrillo [ 34 ] presents two methods to reconstruct the trajectory of occluded markers in motion capture sessions of guitarists plucking strings: “a rigid-body model to track the motion of the guitar strings and a flexible-body model to track the motion of the hands.” Armondes et al [ 35 ] propose a multimodal approach to the strumming technique using multichannel audio recording, an IMU with six degrees of freedom, and a high frame rate video recording to analyze the connection between gestures, rhythm, and generated sound.…”

MarkerLess Motion Capture: ML-MoCap, a low-cost modular multi-camera setup

“…For example, during finger movements, markers typically move 0.55 mm for each 10º of flexion around either the proximal interphalangeal (PIP) or the distal interphalangeal (DIP) joints [5]. Furthermore, participants move less naturally with markers attached [6]. Ideally, motion capture systems allow for measurements of natural movements under minimal environmental constraints and without labour-intensive postprocessing.…”

“…Recent results indicate that OpenPose-based markerless motion capture achieves positional errors smaller than 30 mm for full-body movements [11]. Elaborate postprocessing can reconstruct occluded markers or correct the joint centres [6]. Nevertheless, marker-based systems remain the trusted tool in clinical and scientific settings due to their superior accuracy.…”