Virtual reality system for stroke recovery for upper limbs using ArUco markers

Lupu, Robert Gabriel; Herghelegiu, Paul; Botezatu, Nicolae; Moldoveanu, Alin; Ferche, Oana; Ilie, Catalin; Levinta, Ana-Maria

doi:10.1109/icstcc.2017.8107092

Cited by 9 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also found similar works indicates that tracking human movements becomes vital and necessary in rehabilitation scheme and the avatar upper-limbs control method are gradually applied in order to extract objective and accurate information. For example, Lupu et al [25] proposed an affordable system with motion tracking by placing markers on limbs for stroke recovery. Tsekleves et al [26] developed and assessed an interactive game-based rehabilitation tool for balance training of adults with neurological injury using Microsoft Kinect sensor.…”

Section: Discussionmentioning

confidence: 99%

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

Cha¹,

Wang

Yan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Backgrounds: Transferring behaviors of a human’s upper-limbs to an avatar is widely used in the field of virtual reality rehabilitation. To realize the transfer, movement tracking technology is required. Traditionally, wearable tracking devices are used to do the tracking, however the devices are expensive and cumbersome. Recently, non-wearable upper-limb tracking solutions are proposed, which are cheaper and more comfortable to interact. But most of the existing products cannot track full upper-limbs including both the arms and all the fingers, which limits the motion paradigm and further may lead to limited rehabilitation effect. Methods: In this paper, a novel method was first proposed for full avatar’s upper-limb control which integrates the fine finger motion and the arm wide range motion. Then, based on the method, a Virtual Reality Rehabilitation System (VRRS) was developed for upper-limb rehabilitation. To test the performance of VRRS, two experiments were designed. First, in order to investigate the effect of VRRS on virtual body ownership, agency, location of the body and usability, we compared it with the partial upper-limb tracking method based on Leap Motion controller (LP) in same virtual environments. Then, to study the feasibility of VRRS in rehabilitation, we recruited 16 stroke patients and split them into two groups: the experimental group and the control group. Each group consisted 8 patients, with and without employing VRRS respectively.Results: The control of full avatar’s upper-limbs improved the users’ senses on body ownership, agency and location of the body. The users preferred to use VRRS. In addition, although the upper-limb motor function of patients from both groups were improved, the difference between the FM scores tested on the first day and the last day of the experimental group was more significant than that of the control group. Conclusions: VRRS based on the proposed method for full avatar’s upper-limbs control was developed, which improved the user experience on embodiment and effectively improved the rehabilitation effect for upper-limbs of stroke patients.Trial registration:The study was registered at the First Affiliated Hospital of Jinan University Identifier: KY-2020-036; Date of registration - June 01, 2020.

show abstract

Section: Discussionmentioning

confidence: 99%

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

Cha¹,

Wang

Yan³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For our collection process, we chose ArUco markers, which are synthetic square markers with a black border and a unique binary (black and white) inner matrix that determines its unique identifier (ID) [ 22 ]. These markers have been used for robotics [ 31 , 32 ], autonomous systems [ 33 ], and virtual reality [ 34 ] thanks to their robustness and versatility. Each detected marker provides the ID and pixel coordinates of its four corners.…”

Section: Methodsmentioning

confidence: 99%

SpeakingFaces: A Large-Scale Multimodal Dataset of Voice Commands with Visual and Thermal Video Streams

Abdrakhmanova

Kuzdeuov

Jarju

et al. 2021

Sensors

View full text Add to dashboard Cite

We present SpeakingFaces as a publicly-available large-scale multimodal dataset developed to support machine learning research in contexts that utilize a combination of thermal, visual, and audio data streams; examples include human–computer interaction, biometric authentication, recognition systems, domain transfer, and speech recognition. SpeakingFaces is comprised of aligned high-resolution thermal and visual spectra image streams of fully-framed faces synchronized with audio recordings of each subject speaking approximately 100 imperative phrases. Data were collected from 142 subjects, yielding over 13,000 instances of synchronized data (∼3.8 TB). For technical validation, we demonstrate two baseline examples. The first baseline shows classification by gender, utilizing different combinations of the three data streams in both clean and noisy environments. The second example consists of thermal-to-visual facial image translation, as an instance of domain transfer.

show abstract

“…It should be noted that two of the studies outlined in Table 2 have prior studies, relating to the same system, that appeared in the search. These were Rehabot (Afyouni et al, 2017;Afyouni et al, 2019a;Afyouni et al, 2019b, Afyouni et al (2020 and TRAVEE (Caraiman et al, 2015;Ferche et al, 2015;Ferche et al, 2017a;Ferche et al, 2017b;Lupu et al, 2017;Ferche et al, 2018;Lupu et al, 2018;Moldoveanu et al, 2019;Petrescu et al, 2020).…”

Section: Study Selectionmentioning

confidence: 99%

Interactions with virtual therapists during motor rehabilitation in immersive virtual environments: a systematic review

Crowe,

Yousefi,

Shahri

et al. 2024

Front. Virtual Real.

View full text Add to dashboard Cite

Introduction: Virtual Reality applications for rehabilitation are increasing in popularity. In traditional as well as virtual rehabilitation, the guidance of an occupational therapist through physical and verbal interaction is often required. Several studies have attempted to implement a virtual therapist or assistant in immersive virtual environments for rehabilitation.Objective: This research aims to systematically review these studies, understanding the therapist’s role and how they are represented in the virtual environment, how the virtual therapist and patient interact and the experience of users. Our goal is to provide guidance for virtual therapist implementations into fully immersive virtual reality environments.Method: The researchers systematically reviewed the literature on virtual therapists in immersive virtual environments for motor rehabilitation (protocol prospectively registered with PROSPERO ID: CRD42022357369).Results and Conclusion: Seven studies were identified, with findings showing that virtual therapists are often human-like avatars that guide patients, requiring them to mimic the therapist’s movements. Visual, haptic, or one-direction audio feedback from the therapist to the patient was provided in five studies. The selected studies were all newly developed custom-made systems, with five systems developed using the same game engine. Studies carrying out user testing utilised several methods to understand user experience, evidencing a positive attitude from therapists and motivated and satisfied patients.Future Research: The roles of virtual therapists for motor rehabilitation needs to be explored further in rigorous empirical studies to gather data on patient and therapist user experience. Considerable potential was identified in the development of virtual therapists and therapeutic alliance in the virtual environment. Future systems should reduce manual input from therapists and include personalisation and individualised patient feedback.Systematic Review Registration: [https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=357369], PROSPERO ID [CRD42022357369].

show abstract

Virtual reality system for stroke recovery for upper limbs using ArUco markers

Cited by 9 publications

References 15 publications

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

SpeakingFaces: A Large-Scale Multimodal Dataset of Voice Commands with Visual and Thermal Video Streams

Interactions with virtual therapists during motor rehabilitation in immersive virtual environments: a systematic review

Contact Info

Product

Resources

About