Differences between lower extremity joint running kinetics captured by marker-based and markerless systems were speed dependent

Tang, Hui; Munkasy, Barry; Li, Li

doi:10.1016/j.jshs.2024.01.002

“…Marker-based techniques use motion capture cameras to directly assess the marker 3D positions and are typically more precise in determining keypoint locations [4,9,10].…”

Section: Introductionmentioning

confidence: 99%

Deep Imputation for Skeleton Data (DISK) for Behavioral Science

Rose,

Michaluk,

Blindauer

et al. 2024

Preprint

0

View full text Add to dashboard Cite

Pose estimation methods and motion capture systems have opened doors to quantitative measurements of animal kinematics. However, these methods are not perfect and contain missing data. Our method, Deep Imputation for Skeleton data (DISK), leverages deep learning algorithms to learn dependencies between keypoints and their dynamics to impute missing tracking data. We developed an unsupervised training scheme, which does not rely on manual annotations, and tested several neural network architectures for the imputation task. We found that transformer outperforms other architectures including graph convolutional networks that were developed specifically for skeleton-based action recognition. We demonstrate the usability and performance of our imputation method on seven different animal skeletons including two multi-animal set-ups. With an optional estimated imputation error, DISK enables behavior scientists to assess the reliability of the imputed data. The imputed recordings allow to detect more episodes of motion, such as steps, and to obtain more statistically robust results when comparing these episodes between experimental conditions. While animal behavior experiments are expensive and complex, tracking errors make sometimes large portions of the experimental data unusable. DISK allows for filling in the missing information and for taking full advantage of the rich behavioral data. This stand-alone imputation package, freely available at https://github.com/bozeklab/DISK.git, is applicable to results of any tracking method (marker-based or markerless) and allows for any type of downstream analysis.

show abstract

“…Some pose estimation systems, in addition to estimating each keypoint position in a given video frame, output an associated confidence score thus allowing keypoint filtering [3][4][5][6][7][8]. Marker-based techniques use motion capture cameras to directly assess the marker 3D positions and are typically more precise in determining keypoint locations [4,9,10]. However marker-based techniques still fail to detect all data points due to occlusions 2 or poor quality triangulation.…”

Section: Introductionmentioning

confidence: 99%

Deep Imputation for Skeleton Data (DISK) for Behavioral Science

Rose,

Michaluk,

Blindauer

et al. 2024

Preprint

0

View full text Add to dashboard Cite

Pose estimation methods and motion capture systems have opened doors to quantitative measurements of animal kinematics. However, these methods are not perfect and contain missing data. Our method, Deep Imputation for Skeleton data (DISK), leverages deep learning algorithms to learn dependencies between keypoints and their dynamics to impute missing tracking data. We developed an unsupervised training scheme, which does not rely on manual annotations, and tested several neural network architectures for the imputation task. We found that transformer outperforms other architectures including graph convolutional networks that were developed specifically for skeleton-based action recognition. We demonstrate the usability and performance of our imputation method on seven different animal skeletons including two multi-animal set-ups. With an optional estimated imputation error, DISK enables behavior scientists to assess the reliability of the imputed data. The imputed recordings allow to detect more episodes of motion, such as steps, and to obtain more statistically robust results when comparing these episodes between experimental conditions. While animal behavior experiments are expensive and complex, tracking errors make sometimes large portions of the experimental data unusable. DISK allows for filling in the missing information and for taking full advantage of the rich behavioral data. This stand-alone imputation package, freely available at https://github.com/bozeklab/DISK.git, is applicable to results of any tracking method (marker-based or markerless) and allows for any type of downstream analysis.

show abstract