Sit-to-Stand Analysis in the Wild Using Silhouettes for Longitudinal Health Monitoring

Masullo, Alessandro; Burghardt, Tilo; Perrett, Toby; Damen, Dima; Mirmehdi, Majid

doi:10.1007/978-3-030-27272-2_15

Cited by 7 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While this hypothesis is valid for ReID of pedestrians, it does not apply to indoor scenarios, where smaller environments do not allow for full development of gait sequences. Moreover, as we showed in [9], patients undergoing physically-impairing surgery will show drastic changes in their mobility. Such changes completely violate the main hypothesis of motion-based ReID algorithms, rendering them inapplicable to long-term clinical monitoring.…”

Section: Reid From Imagesmentioning

confidence: 68%

“…Due to their light weight representation, silhouettes are often the preferred form of data in Internet of Things (IoT) and AAL applications [7]. In our previous works, we demonstrated that silhouettes can be reliably employed for long-term home monitoring applications to measure important health-related parameters, for example, measurement of calorie expenditure [8] and the speed of transition from sitting to standing or standing to sitting (StS) [9], which are proxy measurements for sedentary behaviour, musculoskeletal illnesses, fall history and many other health-related conditions. Previous work in the field also showed that silhouettes can be successfully employed for fall detection [10] and abnormal gait analysis [11].…”

Section: Introductionmentioning

confidence: 99%

“…We consider real-life monitoring scenarios and we tackle the rather complex case where only the monitored participants wear an accelerometer whilst being visually recorded amongst other persons, as Figure 1 illustrates. Moreover, our pioneering solution to the Video-Acceleration matching problem can operate on even short (≈3 s) video snippets, so that quick and clinically relevant movements (e.g., Sit-to-Stand [9]) can be associated to a specific individual in spite of the length of the event. We propose a novel multimodal deep-learning detection framework that maps video silhouettes and accelerations into a latent space where the Euclidean distance can be measured to match video and acceleration streams.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring Applications

Masullo

Burghardt

Damen

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

The use of visual sensors for monitoring people in their living environments is critical in processing more accurate health measurements, but their use is undermined by the issue of privacy. Silhouettes, generated from RGB video, can help towards alleviating the issue of privacy to some considerable degree. However, the use of silhouettes would make it rather complex to discriminate between different subjects, preventing a subject-tailored analysis of the data within a free-living, multi-occupancy home. This limitation can be overcome with a strategic fusion of sensors that involves wearable accelerometer devices, which can be used in conjunction with the silhouette video data, to match video clips to a specific patient being monitored. The proposed method simultaneously solves the problem of Person ReID using silhouettes and enables home monitoring systems to employ sensor fusion techniques for data analysis. We develop a multimodal deep-learning detection framework that maps short video clips and accelerations into a latent space where the Euclidean distance can be measured to match video and acceleration streams. We train our method on the SPHERE Calorie Dataset, for which we show an average area under the ROC curve of 76.3% and an assignment accuracy of 77.4%. In addition, we propose a novel triplet loss for which we demonstrate improving performances and convergence speed.

show abstract

Section: Reid From Imagesmentioning

confidence: 68%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring Applications

Masullo

Burghardt

Damen

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…1). Two STS parameters were quantified: firstly, the speed of ascent (SOA) was evaluated similarly to our previous work [8] by looking at the maximum derivative of the head trajectory within each video clip; secondly, we estimated the STS "final attempt duration." This process follows the previous estimation of the speed of ascent and the moment in time, namely t SOA , at which the measure is detected.…”

Section: Automatic Detection and Quantification Methodologymentioning

confidence: 99%

Automated Real-World Video Analysis of Sit-to-Stand Transitions Predicts Parkinson’s Disease Severity

Morgan,

Masullo,

Mirmehdi

et al. 2023

Digit Biomark

Self Cite

View full text Add to dashboard Cite

Introduction: Technology holds the potential to track disease progression and response to neuroprotective therapies in Parkinson’s disease (PD). The sit-to-stand (STS) transition is a frequently occurring event which is important to people with PD. The aim of this study was to demonstrate an automatic approach to quantify STS duration and speed using a real-world free-living dataset and look at clinical correlations of the outcomes, including whether STS parameters change when someone withholds PD medications. Methods: Eighty-five hours of video data were collected from 24 participants staying in pairs for 5-day periods in a naturalistic setting. Skeleton joints were extracted from the video data; the head trajectory was estimated and used to estimate the STS parameters of duration and speed. Results: 3.14 STS transitions were seen per hour per person on average. Significant correlations were seen between automatic and manual STS duration (Pearson rho − 0.419, p = 0.042) and between automatic STS speed and manual STS duration (Pearson rho − 0.780, p < 0.001). Significant and strong correlations were seen between the gold-standard clinical rating scale scores and both STS duration and STS speed; these correlations were not seen in the STS transitions when the participants were carrying something in their hand(s). Significant differences were seen at the cohort level between control and PD participants’ ON medications’ STS duration (U = 6,263, p = 0.018) and speed (U = 9,965, p < 0.001). At an individual level, only two participants with PD became significantly slower to STS when they were OFF medications; withholding medications did not significantly change STS duration at an individual level in any participant. Conclusion: We demonstrate a novel approach to automatically quantify and ecologically validate two STS parameters which correlate with gold-standard clinical tools measuring disease severity in PD.

show abstract

“…This study considers real time-series data generated by SPHERE systems monitoring patient activity in the home before and after total hip or knee replacement. The types of data available from the SPHERE system in each home include metrics derived from Bluetooth-based indoor localization of the patient [ 18 ], continuous estimation of posture and ambulatory activities using a wrist-worn accelerometer [ 24 ], and silhouette data generated using a depth-sensing video camera [ 25 ]. Although the overall system was developed by SPHERE in the absence of equivalent commercial systems, the capabilities of such a system would readily be within the reach of several companies in the consumer smart home market.…”

Section: Introductionmentioning

confidence: 99%

Toward Enhanced Clinical Decision Support for Patients Undergoing a Hip or Knee Replacement: Focus Group and Interview Study With Surgeons

Grant¹,

Tonkin²,

Craddock³

et al. 2023

JMIR Perioper Med

View full text Add to dashboard Cite

Background The current assessment of recovery after total hip or knee replacement is largely based on the measurement of health outcomes through self-report and clinical observations at follow-up appointments in clinical settings. Home activity-based monitoring may improve assessment of recovery by enabling the collection of more holistic information on a continuous basis. Objective This study aimed to introduce orthopedic surgeons to time-series analyses of patient activity data generated from a platform of sensors deployed in the homes of patients who have undergone primary total hip or knee replacement and understand the potential role of these data in postoperative clinical decision-making. Methods Orthopedic surgeons and registrars were recruited through a combination of convenience and snowball sampling. Inclusion criteria were a minimum required experience in total joint replacement surgery specific to the hip or knee or familiarity with postoperative recovery assessment. Exclusion criteria included a lack of specific experience in the field. Of the 9 approached participants, 6 (67%) orthopedic surgeons and 3 (33%) registrars took part in either 1 of 3 focus groups or 1 of 2 interviews. Data were collected using an action-based approach in which stimulus materials (mock data visualizations) provided imaginative and creative interactions with the data. The data were analyzed using a thematic analysis approach. Results Each data visualization was presented sequentially followed by a discussion of key illustrative commentary from participants, ending with a summary of key themes emerging across the focus group and interview data set. Conclusions The limitations of the evidence are as follows. The data presented are from 1 English hospital. However, all data reflect the views of surgeons following standard national approaches and training. Although convenience sampling was used, participants’ background, skills, and experience were considered heterogeneous. Passively collected home monitoring data offered a real opportunity to more objectively characterize patients’ recovery from surgery. However, orthopedic surgeons highlighted the considerable difficulty in navigating large amounts of complex data within short medical consultations with patients. Orthopedic surgeons thought that a proposed dashboard presenting information and decision support alerts would fit best with existing clinical workflows. From this, the following guidelines for system design were developed: minimize the risk of misinterpreting data, express a level of confidence in the data, support clinicians in developing relevant skills as time-series data are often unfamiliar, and consider the impact of patient engagement with data in the future. International Registered Report Identifier (IRRID) RR2-10.1136/bmjopen-2018-021862

show abstract

Sit-to-Stand Analysis in the Wild Using Silhouettes for Longitudinal Health Monitoring

Cited by 7 publications

References 19 publications

Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring Applications

Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring Applications

Automated Real-World Video Analysis of Sit-to-Stand Transitions Predicts Parkinson’s Disease Severity

Toward Enhanced Clinical Decision Support for Patients Undergoing a Hip or Knee Replacement: Focus Group and Interview Study With Surgeons

Contact Info

Product

Resources

About