Identifying Hand Use and Hand Roles After Stroke Using Egocentric Video

Abstract

Objective: Upper limb (UL) impairment impacts quality of life, but is common after stroke. UL function evaluated in the clinic may not reflect use in activities of daily living (ADLs) after stroke, and current approaches for assessment at home rely on self-report and lack details about hand function. Wrist-worn accelerometers have been applied to capture UL use, but also fail to reveal details of hand function. In response, a wearable system is proposed consisting of egocentric cameras combined with computer vision approaches, in order to identify hand use (hand-object interactions) and the role of the more-affected hand (as stabilizer or manipulator) in unconstrained environments. Methods: Nine stroke survivors recorded their performance of ADLs in a home simulation laboratory using an egocentric camera. Motion, hand shape, colour, and hand size change features were generated and fed into random forest classifiers to detect hand use and classify hand roles. Leave-one-subject-out cross-validation (LOSOCV) and leave-one-task-out cross-validation (LOTOCV) were used to evaluate the robustness of the algorithms. Results: LOSOCV and LOTOCV F1-scores for more-affected hand use were 0.64 ± 0.24 and 0.76 ± 0.23, respectively. For less-affected hands, LOSOCV and LOTOCV F1-scores were 0.72 ± 0.20 and 0.82 ± 0.22. F1-scores for hand role classification were 0.70 ± 0.19 and 0.68 ± 0.23 in the more-affected hand for LOSOCV and LOTOCV, respectively, and 0.59 ± 0.23 and 0.65 ± 0.28 in the less-affected hand. Conclusion: The results demonstrate the feasibility of predicting hand use and the hand roles of stroke survivors from egocentric videos.

Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test

“…In the CMT framework that we built, we use two markers: red chopsticks markers and blue finger markers. We perform marker detection on the index finger because it is visible in egocentric vision [6], while the middle finger is closed. A red marker is attached to the chopsticks for finger joint angle estimation by obtaining ground truth data to minimize errors.…”

“…This decline is primarily due to age-related degenerative changes in the musculoskeletal, vascular, and nervous systems [5]. This is often followed by several pathological conditions such as stroke [6] and other neurological diseases. Researchers have examined whether chopsticks manipulation among stroke patients can increase levels of eating independence [7].…”

“…The advantage of egocentric vision is reduced user privacy concerns [15], the ability to monitor people when they are mobile [16], and finding attention during activities [17]. Currently, egocentric vision research has begun to be developed for hand rehabilitation [6]. Unlike egocentric vision, visual attention refers to maintaining the awareness of visual scene's specific locations, objects, or attributes.…”

A Framework for Sensor-Based Assessment of Upper-Limb Functioning in Hemiparesis

“…Wrist-worn Accelerometers (Uswatte et al, 2006a;Bailey et al, 2014Bailey et al, , 2015 Wrist-worn Inertial measurement units (Bochniewicz et al, 2017;de Lucena et al, 2017;David et al, 2020;Lum et al, 2020) Finger-worn magnetic ring (Friedman et al, 2014) Egocentric camera (Nguyen et al, 2016;Tsai et al, 2020Tsai et al, , 2021 Upper-limb use (ui)…”

Comparing algorithms for assessing upper limb use with inertial measurement units

“…The shortcomings of current questionnairebased assessments of upper limb use in daily life [1] have led to a surge in the use of wearable sensors for this purpose. Several research groups have explored different sensing modalities [2]- [9] and data analysis techniques [6], [8], [10]- [13] for assessing the amount and quality of upper limb use outside the clinic.…”