Human Interaction Smart Subsystem—Extending Speech-Based Human-Robot Interaction Systems with an Implementation of External Smart Sensors

Abstract

This paper presents a more detailed concept of Human-Robot Interaction systems architecture. One of the main differences between the proposed architecture and other ones is the methodology of information acquisition regarding the robot's interlocutor. In order to obtain as much information as possible before the actual interaction took place, a custom Internet-of-Things-based sensor subsystems connected to Smart Infrastructure was designed and implemented, in order to support the interlocutor identification and acquisition of initial interaction parameters. The Artificial Intelligence interaction framework of the developed robotic system (including humanoid Pepper with its sensors and actuators, additional local, remote and cloud computing services) is being extended with the use of custom external subsystems for additional knowledge acquisition: device-based human identification, visual identification and audio-based interlocutor localization subsystems. These subsystems were deeply introduced and evaluated in this paper, presenting the benefits of integrating them into the robotic interaction system. In this paper a more detailed analysis of one of the external subsystems-Bluetooth Human Identification Smart Subsystem-was also included. The idea, use case, and a prototype, integration of elements of Smart Infrastructure systems and the prototype implementation were performed in a small front office of the Weegree company as a decent test-bed application area.The days, when simply having a robot was impressive, are gone. The technology should be more than just presenting content. It should be truly interactive. The authors of this work evaluated methods for the purpose of measurement and influencing the Human-Robot interaction User Experience, (see Reference [1]), concluding that the robot's interaction is perceived as more natural if the robot is able to personalize its communication content towards a particular interlocutor. It became clear, that the system should be able to get as much information as possible during and before the conversation takes place [2]. One of the key aspects in a healthy conversation is emotional state recognition. All the improvements of the robots presented in this paper are aiming for natural, real integration with human customers with the implementation of Artificial Intelligence [3,4].The authors of this work have indicated extending the robot's capabilities by adding additional external wireless sensors in Reference [2], however without Smart Infrastructure. Implementation of a remote PID (Passive Infrared Detector) sensor in order to get a signal "someone is coming" seemed to be enough at that stage.In this paper an extended approach is being presented, introducing Human Identification Smart Subsystems (HISS)-a set of external sensors, which enable us to obtain much more valuable information, resulting from the communication with the Smart Infrastructure [5] or databases of the customer. The idea presented in this paper has been successfully verified and implemented and the con...

Expectations and Perceptions of Healthcare Professionals for Robot Deployment in Hospital Environments During the COVID-19 Pandemic

“…Different social robots are being adapted and applied to the telemedicine concept with significant growth in this period (Khan et al, 2020 ). Renowned platforms, such as Pepper (SoftBank Robotics, Tokyo, Japan) heads the list of robots to be called to keep the patient-clinician communication, even without representing a contagion risk (Podpora et al, 2020 ). This task is not unknown by the platform, since several studies have shown the potential of Pepper working in this application (Pandey and Gelin, 2018 ; Stock and Merkle, 2018 ).…”

Trends, Technologies, and Key Challenges in Smart and Connected Healthcare

“…They can also be used to deliver other goods and groceries to avoid visits to supermarkets and reduce potential human physical interactions. An example of a disinfectant drone was that developed by DJI [208], a Chinese company; this drone can disinfect one hundred square meters in one hour. These drones were used in Spain as well [172].…”

Robotic Sensing and Systems for Smart Cities

“…The study in [ 6 ] presents an integrated human–robot interaction system architecture. One of the main differences between the proposed architecture and other ones is the information acquisition method regarding the robot’s interlocutor.…”