Extending Human–Robot Relationships Based in Music With Virtual Presence

McCallum, Louis; McOwan, Peter W.

doi:10.1109/tcds.2017.2779218

Cited by 2 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A humanoid robot's visual imitation of 3-D motion of a human is developed by a neural-network-based inverse kinematics [2] or support vector machine for the classification of 11 low-body postures [4]. In [3], the suggested robot Mortimer including social behaviors can increase engagement and social presence; in addition, the effect of extending weekly collocated musical improvisation sessions is investigated by making Mortimer an active member of the participant's virtual social network. In [5], specific human following through machine learning of SSD-FN-KCF is developed.…”

Section: Introductionmentioning

confidence: 99%

Human–Robot Collaboration Using Sequential-Recurrent-Convolution-Network-Based Dynamic Face Emotion and Wireless Speech Command Recognitions

Hwang

Deng

2023

IEEE Access

View full text Add to dashboard Cite

The proposed sequential recurrent convolution network (SRCN) includes two parts: one convolution neural network (CNN) and a sequence of long short-term memory (LSTM) models. The CNN is to achieve the feature vector of face emotion or speech command. Then, a sequence of LSTM models with the shared weight reflects a sequence of inputs provided by a (pre-trained) CNN with a sequence of input sub-images or spectrograms corresponding to face emotion and speech command, respectively. Simply put, one SRCN for dynamic face emotion recognition (SRCN-DFER) and another SRCN for wireless speech command recognition (SRCN-WSCR) are developed. The proposed approach not only effectively tackles the recognitions of dynamic mapping of face emotion and speech command with average generalized recognition rate of 98% and 96.7% but also prevents the overfitting problem in a noisy environment. The comparisons among mono and stereo visions, Deep CNN, and ResNet50 confirm the superiority of the proposed SRCN-DFER. The comparisons among SRCN-WSCR with noise-free data, SRCN-WSCR with noisy data, and multiclass support vector machine validate its robustness. Finally, the human-robot collaboration (HRC) using our developed omnidirectional service robot, including human and face detections, trajectory tracking by the previously designed adaptive stratified finite-time saturated control, face emotion and speech command recognitions, and music play, validates the effectiveness, feasibility, and robustness of the proposed method.

show abstract

Section: Introductionmentioning

confidence: 99%

Human–Robot Collaboration Using Sequential-Recurrent-Convolution-Network-Based Dynamic Face Emotion and Wireless Speech Command Recognitions

Hwang

Deng

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

Nonverbal Sound in Human-Robot Interaction: A Systematic Review

Zhang¹,

Fitter²

2023

J. Hum.-Robot Interact.

View full text Add to dashboard Cite

Nonverbal sound offers great potential to enhance robots’ interactions with humans, and a growing body of research has begun to explore nonverbal sound for tasks such as sound source localization, explicit communication, and improving sociability. However, nonverbal sound has a broad interpretation and design space that can draw from areas such as machine learning, music theory, and foley. We sought to identify and compare use cases and approaches for nonverbal sound in human-robot interaction through a systematic review. A search of sound and robotics-related publisher databases yielded 148 peer-reviewed articles presenting systems, studies, and taxonomies. Differences in taxonomy and overlap of terminology with adjacent research fields such as speech, gaze, and gesture posed difficulties for the search, which we attempted to address through a multi-stage search process. Based on the reviewed articles, we developed a pair of taxonomies using scientific communication principles and analyzed study designs and measures for the creation of nonverbal robot sound. We discuss recommendations for the field, including the use of the new taxonomies; methods for design, generation, and validation; and paths for future research. Roboticists may benefit from incorporating nonverbal sound as a key component in multimodal human-robot interaction.

show abstract

Extending Human–Robot Relationships Based in Music With Virtual Presence

Cited by 2 publications

References 20 publications

Human–Robot Collaboration Using Sequential-Recurrent-Convolution-Network-Based Dynamic Face Emotion and Wireless Speech Command Recognitions

Human–Robot Collaboration Using Sequential-Recurrent-Convolution-Network-Based Dynamic Face Emotion and Wireless Speech Command Recognitions

Nonverbal Sound in Human-Robot Interaction: A Systematic Review

Contact Info

Product

Resources

About