Using Computer Vision to Annotate Video-Recoded Direct Observation of Physical Behavior

Keadle, Sarah K.; Eglowski, Skylar; Ylarregui, Katie; Strath, Scott J.; Martinez, Julian; Dekhtyar, Alex; Kagan, Vadim

doi:10.3390/s24072359

Cited by 4 publications

(1 citation statement)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning, a part of machine learning, is effective for analyzing complex data like walking videos. It is excellent at finding patterns in video data, making it ideal for postural control analysis [ 8 , 9 , 10 ]. Deep learning techniques, such as Openpose, have been instrumental in advancing postural control analysis in gait assessment.…”

Section: Introductionmentioning

confidence: 99%

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

Lee,

Chen,

Lin

et al. 2024

Bioengineering

View full text Add to dashboard Cite

Noninvasive tracking devices are widely used to monitor real-time posture. Yet significant potential exists to enhance postural control quantification through walking videos. This study advances computational science by integrating OpenPose with a Support Vector Machine (SVM) to perform highly accurate and robust postural analysis, marking a substantial improvement over traditional methods which often rely on invasive sensors. Utilizing OpenPose-based deep learning, we generated Dynamic Joint Nodes Plots (DJNP) and iso-block postural identity images for 35 young adults in controlled walking experiments. Through Temporal and Spatial Regression (TSR) models, key features were extracted for SVM classification, enabling the distinction between various walking behaviors. This approach resulted in an overall accuracy of 0.990 and a Kappa index of 0.985. Cutting points for the ratio of top angles (TAR) and the ratio of bottom angles (BAR) effectively differentiated between left and right skews with AUC values of 0.772 and 0.775, respectively. These results demonstrate the efficacy of integrating OpenPose with SVM, providing more precise, real-time analysis without invasive sensors. Future work will focus on expanding this method to a broader demographic, including individuals with gait abnormalities, to validate its effectiveness across diverse clinical conditions. Furthermore, we plan to explore the integration of alternative machine learning models, such as deep neural networks, enhancing the system’s robustness and adaptability for complex dynamic environments. This research opens new avenues for clinical applications, particularly in rehabilitation and sports science, promising to revolutionize noninvasive postural analysis.

show abstract

Section: Introductionmentioning

confidence: 99%

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

Lee,

Chen,

Lin

et al. 2024

Bioengineering

View full text Add to dashboard Cite

show abstract

The Application of Knowledge Graph Convolutional Network-Based Film and Television Interaction Under Artificial Intelligence

Yang,

Li,

Jahng

2024

IEEE Access

View full text Add to dashboard Cite

Using Video Cameras to Assess Physical Activity and Other Well-Being Behaviors in Urban Environments: Feasibility, Reliability, and Participant Reactivity Studies

Benton,

Evans,

Anderson

et al. 2024

JMIR Public Health Surveill

View full text Add to dashboard Cite

Background Unobtrusive observation is a promising method for assessing physical activity and other well-being behaviors (eg, social interactions) in urban environments, without participant burden and biases associated with self-report. However, current methods require multiple in-person observers. Using video cameras instead could allow for more accurate observations at lower cost and with greater flexibility in scheduling. Objective This research aimed to test the feasibility of using stationary wireless video cameras to observe physical activity and other well-being behaviors, and to assess its reliability and potential participant reactivity. Methods Across 3 cross-sectional studies, 148 hours of video recordings were collected from 6 outdoor public spaces in Manchester, United Kingdom. The videos were coded by 3 researchers using MOHAWk (Method for Observing Physical Activity and Wellbeing)—a validated in-person observation tool for assessing physical activity, social interactions, and people taking notice of the environment. Inter- and intrarater reliabilities were assessed using intraclass correlation coefficients (ICCs). Intercept surveys were conducted to assess public awareness of the cameras and whether they altered their behavior due to the presence of cameras. Results The 148 hours of video recordings were coded in 85 hours. Interrater reliability between independent coders was mostly “excellent” (ICCs>0.90; n=36), with a small number of “good” (ICCs>0.75; n=2), “moderate” (ICCs=0.5-0.75; n=3), or “poor” (ICCs<0.5; n=1) ICC values. Reliability decreased at night, particularly for coding ethnic group and social interactions, but remained mostly “excellent” or “good.” Intrarater reliability within a single coder after a 2-week interval was “excellent” for all but 1 code, with 1 “good” ICC value for assessing vigorous physical activity, indicating that the coder could reproduce similar results over time. Intrarater reliability was generally similar during the day and night, apart from ICC values for coding ethnic group, which reduced from “excellent” to “good” at night. Intercept surveys with 86 public space users found that only 5 (5.8%) participants noticed the cameras used for this study. Importantly, all 5 said that they did not alter their behavior as a result of noticing these cameras, therefore, indicating no evidence of reactivity. Conclusions Camera-based observation methods are more reliable than in-person observations and do not produce participant reactivity often associated with self-report methods. This method requires less time for data collection and coding, while allowing for safe nighttime observation without the risk to research staff. This research is a significant first step in demonstrating the potential for camera-based methods to improve natural experimental studies of real-world environmental interventions. It also provides a rigorous foundation for developing more scalable automated computer vision algorithms for assessing human behaviors.

show abstract

Using Computer Vision to Annotate Video-Recoded Direct Observation of Physical Behavior

Cited by 4 publications

References 57 publications

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

The Application of Knowledge Graph Convolutional Network-Based Film and Television Interaction Under Artificial Intelligence

Using Video Cameras to Assess Physical Activity and Other Well-Being Behaviors in Urban Environments: Feasibility, Reliability, and Participant Reactivity Studies

Contact Info

Product

Resources

About