Motion Tracker: Camera-Based Monitoring of Bodily Movements Using Motion Silhouettes

Abstract

Researchers in the cognitive and affective sciences investigate how thoughts and feelings are reflected in the bodily response systems including peripheral physiology, facial features, and body movements. One specific question along this line of research is how cognition and affect are manifested in the dynamics of general body movements. Progress in this area can be accelerated by inexpensive, non-intrusive, portable, scalable, and easy to calibrate movement tracking systems. Towards this end, this paper presents and validates Motion Tracker, a simple yet effective software program that uses established computer vision techniques to estimate the amount a person moves from a video of the person engaged in a task (available for download from http://jakory.com/motion-tracker/). The system works with any commercially available camera and with existing videos, thereby affording inexpensive, non-intrusive, and potentially portable and scalable estimation of body movement. Strong between-subject correlations were obtained between Motion Tracker’s estimates of movement and body movements recorded from the seat (r =.720) and back (r = .695 for participants with higher back movement) of a chair affixed with pressure-sensors while completing a 32-minute computerized task (Study 1). Within-subject cross-correlations were also strong for both the seat (r =.606) and back (r = .507). In Study 2, between-subject correlations between Motion Tracker’s movement estimates and movements recorded from an accelerometer worn on the wrist were also strong (rs = .801, .679, and .681) while people performed three brief actions (e.g., waving). Finally, in Study 3 the within-subject cross-correlation was high (r = .855) when Motion Tracker’s estimates were correlated with the movement of a person’s head as tracked with a Kinect while the person was seated at a desk (Study 3). Best-practice recommendations, limitations, and planned extensions of the system are discussed.

Beyond Dyadic Coordination: Multimodal Behavioral Irregularity in Triads Predicts Facets of Collaborative Problem Solving

“…We used the validated Motion Tracker software (Kory, D'Mello, & Olney, 2015), which applies background subtraction and image silhouetting algorithms to detect the amount of body movement of each participant from his or her webcam data (at 25 Hz). Motion Tracker has demonstrated strong correlations with movement estimates derived from wrist accelerometers, Microsoft Kinect videos, and high-fidelity seat and back movement pressure sensors (Kory et al, 2015). Webcam videos were formatted as MP4 files and included participants' upper torso and head, though participants could lean forward or backward in their chair, sometimes obscuring the upper torso.…”

Elderly Body Movement Alteration at 2nd Experience of Digital Art Installation with Cognitive and Motivation Scores

“…Body motion analysis has been previously used to understand psycho-cognitive reactions in seated people in several contexts, including head movements in autistic children in social contexts [15], adult body posture variation during Stroop test performance [16], and use of a computerized tutor that simulates discourse patterns and pedagogical strategies of a typical human tutor [17].…”

The Human Touch: Using a Webcam to Autonomously Monitor Compliance During Visual Field Assessments

“…However, even in the present work, the association between blink rate and threshold variability was significant only in healthy eyes (not glaucoma), and in general no single biomarker was strongly associated with overall test-retest measurement error-with a more robust association being observed when multiple biomarkers were combined together. Outside of ophthalmology, the present study is also not the first to examine eye movements, 47 head movements, 48 or facial expressions 49,50 as ways of determining whether an individual is alert and engaged.…”