Skeleton-based visualization of poor body movements in a child's gross-motor assessment using convolutional auto-encoder

Suzuki, Satoshi; Amemiya, Yoshihito; Sato, Maiko

doi:10.1109/icm46511.2021.9385618

Cited by 7 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• The definition of anomalous human behaviours can differ across applications. While most of the existing papers focused on detecting anomalous human behaviours in general, four papers focused on detecting anomalous behaviours for specific applications, that is, drunk walking [23], poor body movements in children [24], abnormal pedestrian behaviours at grade crossings [25] and crimebased anomalies [7]. Further, the nature of anomalous behaviours can vary depending upon various factors, like span of time, crowded scenes, and specific actionbased anomalies.…”

Section: Discussionmentioning

confidence: 99%

“…The authors used average of recall and specificity to evaluate the models due to the unbalanced dataset and found that occlusion-aware input achieved the highest results. Suzuki et al [24] trained a Convolutional AE (CAE) on good gross motor movements in children and detected poor limb motion as an anomaly. Motion time-series images [45] were obtained from skeletons estimated from the videos of kindergarten children participants.…”

Section: A Reconstruction Approachesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…In recent years, deep learning methods have been developed to use skeletons for different applications, such as action recognition [43], medical diagnosis [24], and sports analytics [44]. The use of skeletons for anomaly detection in videos is an under-explored area, and concerted research is needed [24]. The human skeletons can help in developing privacypreserving solutions for private dwellings, crowded/public areas, medical settings, rehabilitation centers and long-term care homes to detect anomalous events that impacts health and safety of individuals.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Skeletal Video Anomaly Detection using Deep Learning: Survey, Challenges and Future Directions

Mishra¹,

Mihailidis²,

Khan³

2023

Preprint

View full text Add to dashboard Cite

The existing methods for video anomaly detection mostly utilize videos containing identifiable facial and appearance-based features. The use of videos with identifiable faces raises privacy concerns, especially when used in a hospital or community-based setting. Appearance-based features can also be sensitive to pixel-based noise, straining the anomaly detection methods to model the changes in the background and making it difficult to focus on the actions of humans in the foreground. Structural information in the form of skeletons describing the human motion in the videos is privacy-protecting and can overcome some of the problems posed by appearance-based features. In this paper, we present a survey of privacy-protecting deep learning anomaly detection methods using skeletons extracted from videos. We present a novel taxonomy of algorithms based on the various learning approaches. We conclude that skeleton-based approaches for anomaly detection can be a plausible privacyprotecting alternative for video anomaly detection. Lastly, we identify major open research questions and provide guidelines to address them.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: A Reconstruction Approachesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Skeletal Video Anomaly Detection using Deep Learning: Survey, Challenges and Future Directions

Mishra¹,

Mihailidis²,

Khan³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, this study did not use a previously validated metric such as the K-DST, which may limit the explainability and generalizability of the model. Liu et al [ 22 ] evaluated the gross motor skills of children with autism with an average age of 5 years, and Suzuki et al [ 23 , 24 ] assessed gross motor skills on a video-by-video basis using a deep learning model with behavioral videos of 4- to 5-year-old children. However, since these studies were conducted on children aged ≥4 years, there is a limitation in that they could not validate the model effectiveness in the <3 years age group, where early intervention is expected to be more effective.…”

Section: Introductionmentioning

confidence: 99%

Comprehensive Assessment and Early Prediction of Gross Motor Performance in Toddlers With Graph Convolutional Networks–Based Deep Learning: Development and Validation Study

Chun,

Jang,

Kim

et al. 2024

JMIR Form Res

View full text Add to dashboard Cite

Background Accurate and timely assessment of children’s developmental status is crucial for early diagnosis and intervention. More accurate and automated developmental assessments are essential due to the lack of trained health care providers and imprecise parental reporting. In various areas of development, gross motor development in toddlers is known to be predictive of subsequent childhood developments. Objective The purpose of this study was to develop a model to assess gross motor behavior and integrate the results to determine the overall gross motor status of toddlers. This study also aimed to identify behaviors that are important in the assessment of overall gross motor skills and detect critical moments and important body parts for the assessment of each behavior. Methods We used behavioral videos of toddlers aged 18-35 months. To assess gross motor development, we selected 4 behaviors (climb up the stairs, go down the stairs, throw the ball, and stand on 1 foot) that have been validated with the Korean Developmental Screening Test for Infants and Children. In the child behavior videos, we estimated each child’s position as a bounding box and extracted human keypoints within the box. In the first stage, the videos with the extracted human keypoints of each behavior were evaluated separately using a graph convolutional networks (GCN)–based algorithm. The probability values obtained for each label in the first-stage model were used as input for the second-stage model, the extreme gradient boosting (XGBoost) algorithm, to predict the overall gross motor status. For interpretability, we used gradient-weighted class activation mapping (Grad-CAM) to identify important moments and relevant body parts during the movements. The Shapley additive explanations method was used for the assessment of variable importance, to determine the movements that contributed the most to the overall developmental assessment. Results Behavioral videos of 4 gross motor skills were collected from 147 children, resulting in a total of 2395 videos. The stage-1 GCN model to evaluate each behavior had an area under the receiver operating characteristic curve (AUROC) of 0.79 to 0.90. Keypoint-mapping Grad-CAM visualization identified important moments in each behavior and differences in important body parts. The stage-2 XGBoost model to assess the overall gross motor status had an AUROC of 0.90. Among the 4 behaviors, “go down the stairs” contributed the most to the overall developmental assessment. Conclusions Using movement videos of toddlers aged 18-35 months, we developed objective and automated models to evaluate each behavior and assess each child’s overall gross motor performance. We identified the important behaviors for assessing gross motor performance and developed methods to recognize important moments and body parts while evaluating gross motor performance.

show abstract

Semi-Supervised Skeleton-Based Covert Cheating Detection in Electronic-Exams

Atabay,

Hassanpour

2024

Iran J Sci Technol Trans Electr Eng

View full text Add to dashboard Cite

Skeleton-based visualization of poor body movements in a child's gross-motor assessment using convolutional auto-encoder

Cited by 7 publications

References 21 publications

Skeletal Video Anomaly Detection using Deep Learning: Survey, Challenges and Future Directions

Skeletal Video Anomaly Detection using Deep Learning: Survey, Challenges and Future Directions

Comprehensive Assessment and Early Prediction of Gross Motor Performance in Toddlers With Graph Convolutional Networks–Based Deep Learning: Development and Validation Study

Semi-Supervised Skeleton-Based Covert Cheating Detection in Electronic-Exams

Contact Info

Product

Resources

About