Face mediated human–robot interaction for remote medical examination

Lalitharatne, Thilina Dulantha; Costi, Leone; Hashem, Ryman; Nisky, Ilana; Jack, Rachael E.; Nanayakkara, Thrishantha; Iida, Fumiya

doi:10.1038/s41598-022-16643-z

Cited by 10 publications

(8 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlikely surgery, during palpation the patient is awake and her/his feedback is a fundamental part of the in-person task. Literature studies have indeed proved that feedback signals such as facial expressions [84] and patient's voice [85] strongly improve the users' performance during palpation, to further confirm the importance of reproducing the same experience that the target user would be subjected to during the in-person task. Therefore, it is mandatory to further study physical examination and palpation and to identify and isolate the important features for general practitioners.…”

Section: Discussionmentioning

confidence: 78%

How to define the correct guidelines for enhanced telepresence and task embodiment in remote palpation.

Costi,

Iida

2023

IOP Conf. Ser.: Mater. Sci. Eng.

View full text Add to dashboard Cite

Teleoperated robots have been widely accepted in several fields of medical practice, enhancing human abilities and allowing remote operation. However, such technology has not been able yet to permeate areas such as primary care and physical examination. Such applications strongly rely on the quality of the interaction between doctor and patient, and on its multimodal nature. In order to achieve remote physical examination is thus mandatory to have a good doctor-robot interface, but what does good mean? Ultimately, the goal is for the user to achieve task embodiment, making the remote task feel like the in-person one. Several research groups have proposed a wide variety of interfaces, showcasing largely different methods of control and feedback, because of the absence of design guidelines. In this work, we argue that the ideal interface for a remote task should resemble as close as possible the experience provided by the in-person equivalent, keeping in consideration the nature of the target users. To support our claims, we analyze many remote interfaces and compare them with the respective in-person task. This analysis is not limited to the medical sector, with examples such as remote abdominal surgery, but it expands to all forms of teleoperation, up to nuclear waste handling and avionics.

show abstract

Section: Discussionmentioning

confidence: 78%

How to define the correct guidelines for enhanced telepresence and task embodiment in remote palpation.

Costi,

Iida

2023

IOP Conf. Ser.: Mater. Sci. Eng.

View full text Add to dashboard Cite

show abstract

“…[148][149][150] A face-mediated, human-robot interaction was developed to provide visual feedback in telemedicine (Figure 10a). [35] The tactile sensor array measures the tactile response of a patient during diagnosis and transmits the acquired data to the robot to render the palpation force as a facial expression. The strength and spatial extent of palpation can be analyzed by visualizing the degree of pressure on a tactile map, and the rendered facial expres-sion changes depending on the amount of pressure.…”

Section: Digital Twins For Diagnostics and Therapeuticsmentioning

confidence: 99%

“…tactile sensors and haptic feedback. [34][35][36] The accuracy of surgical operations can be improved by using AR technology to guide the location of surgical instruments. [37,38] Furthermore, AI technology and 5G connectivity facilitate the analysis of acquired healthcare signals more efficiently and solve personalized security issues.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Metaverse Wearables for Immersive Digital Healthcare: A Review

Kim,

Yang,

Lee

et al. 2023

Advanced Science

View full text Add to dashboard Cite

The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all‐encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high‐quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post‐pandemic era.

show abstract

“…Alternatively, virtual and/or physical training simulators have been proposed to address the issues related to SPs. Medical training simulators like the robopatient (Lalitharatne et al, 2021;Lalitharatne et al, 2022;Tan et al, 2022) enable medical students to practice hands-on medical skills on demand, as opposed to on limited resources of professional actors or real patients (Walker et al, 1990). These technologies have been proven to lead to better development of the required skills by medical students (Schubart et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

Vocal pain expression augmentation for a robopatient

Protpagorn,

Lalitharatne,

Costi

et al. 2023

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

Abdominal palpation is one of the basic but important physical examination methods used by physicians. Visual, auditory, and haptic feedback from the patients are known to be the main sources of feedback they use in the diagnosis. However, learning to interpret this feedback and making accurate diagnosis require several years of training. Many abdominal palpation training simulators have been proposed to date, but very limited attempts have been reported in integrating vocal pain expressions into physical abdominal palpation simulators. Here, we present a vocal pain expression augmentation for a robopatient. The proposed robopatient is capable of providing real-time facial and vocal pain expressions based on the exerted palpation force and position on the abdominal phantom of the robopatient. A pilot study is conducted to test the proposed system, and we show the potential of integrating vocal pain expressions to the robopatient. The platform has also been tested by two clinical experts with prior experience in abdominal palpation. Their evaluations on functionality and suggestions for improvements are presented. We highlight the advantages of the proposed robopatient with real-time vocal and facial pain expressions as a controllable simulator platform for abdominal palpation training studies. Finally, we discuss the limitations of the proposed approach and suggest several future directions for improvements.

show abstract

Face mediated human–robot interaction for remote medical examination

Cited by 10 publications

References 54 publications

How to define the correct guidelines for enhanced telepresence and task embodiment in remote palpation.

How to define the correct guidelines for enhanced telepresence and task embodiment in remote palpation.

Metaverse Wearables for Immersive Digital Healthcare: A Review

Vocal pain expression augmentation for a robopatient

Contact Info

Product

Resources

About