Comparing Heads-up, Hands-free Operation of Ground Robots to Teleoperation

Marge, Matthew; Powers, Aaron; Brookshire, Jonathan; Jay, T; Jenkins, Odest Chadwicke; Geyer, Christopher

doi:10.15607/rss.2011.vii.027

Cited by 7 publications

(8 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In many practical applications, a human instructs the UGV to perform certain tasks, such as changing its motion or speed, taking photographs, or making phone calls. These instructions are typically communicated using voice commands (Fritsch et al, 2004), hand gestures (Doisy et al, 2013; Marge et al, 2011), or haptic interfaces (Ghosh et al, 2014; Park and Howard, 2010). Moreover, some smart carts and autonomous luggage robots allow users to interact using smartphone applications (DiGiacomcantonio and Gebreyes, 2014).…”

Section: Categorization Of Autonomous Person-following Behaviorsmentioning

confidence: 99%

Person-following by autonomous robots: A categorical overview

Islam

Hong

Sattar

2019

The International Journal of Robotics Research

111

View full text Add to dashboard Cite

A wide range of human-robot collaborative applications in diverse domains such as manufacturing, health care, the entertainment industry, and social interactions, require an autonomous robot to follow its human companion. Different working environments and applications pose diverse challenges by adding constraints on the choice of sensors, the degree of autonomy, and dynamics of a person-following robot. Researchers have addressed these challenges in many ways and contributed to the development of a large body of literature. This paper provides a comprehensive overview of the literature by categorizing different aspects of person-following by autonomous robots. Also, the corresponding operational challenges are identified based on various design choices for ground, underwater, and aerial scenarios. In addition, state-of-the-art methods for perception, planning, control, and interaction are elaborately discussed and their applicability in varied operational scenarios are presented. Then, some of the prominent methods are qualitatively compared, corresponding practicalities are illustrated, and their feasibility is analyzed for various usecases. Furthermore, several prospective application areas are identified, and open problems are highlighted for future research.

show abstract

Section: Categorization Of Autonomous Person-following Behaviorsmentioning

confidence: 99%

Person-following by autonomous robots: A categorical overview

Islam

Hong

Sattar

2019

The International Journal of Robotics Research

111

View full text Add to dashboard Cite

show abstract

“…The TeamTalk platform used in this work offers a bidirectional method for interpreting commands by detecting problems with task execution, requesting help, and responding to amended instructions from the human. The interaction modality is spoken dialogue; thus, the human is able to interact in a "heads-up, hands-free" way [69]. When disambiguating referring expressions, TeamTalk uses natural-language generation templates to describe its surroundings.…”

Section: Asking For Helpmentioning

confidence: 99%

Miscommunication Detection and Recovery in Situated Human–Robot Dialogue

Marge

Rudnicky

2019

ACM Trans. Interact. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Even without speech recognition errors, robots may face difficulties interpreting natural-language instructions. We present a method for robustly handling miscommunication between people and robots in task-oriented spoken dialogue. This capability is implemented in TeamTalk, a conversational interface to robots that supports detection and recovery from the situated grounding problems of referential ambiguity and impossible actions. We introduce a representation that detects these problems and a nearest-neighbor learning algorithm that selects recovery strategies for a virtual robot. When the robot encounters a grounding problem, it looks back on its interaction history to consider how it resolved similar situations. The learning method is trained initially on crowdsourced data but is then supplemented by interactions from a longitudinal user study in which six participants performed navigation tasks with the robot. We compare results collected using a general model to user-specific models and find that user-specific models perform best on measures of dialogue efficiency, while the general model yields the highest agreement with human judges. Our overall contribution is a novel approach to detecting and recovering from miscommunication in dialogue by including situated context, namely, information from a robot's path planner and surroundings. CCS Concepts: • Computing methodologies → Discourse, dialogue and pragmatics; • Humancentered computing → Natural language interfaces;

show abstract

“…The majority of gesture systems today focus on gesture recognition [20] which is a classification task that does not require the location or orientation of the gesture [27,18]. Often, this recognition is performed in batch, and has a slightly different goal, namely to identify times in a video clip in which certain gestures occur.…”

Section: Related Workmentioning

confidence: 99%

Interpreting multimodal referring expressions in real time

Whitney

Eldon

Oberlin

et al. 2016

2016 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Humans communicate about objects using language, gesture, and context, fusing information from multiple modalities over time. Robots need to interpret this communication in order to collaborate with humans on shared tasks. Processing communicative input incrementally has the potential to increase the speed and accuracy of a robot's reaction. It also enables the robot to incorporate the relative timing of words and gestures into the understanding process. To address this problem, we define a multimodal Bayes filter for interpreting a person's referential expressions to objects. Our approach outputs a distribution over the referent object at 14Hz, updating dynamically as it receives new observations of the person's spoken words and gestures. We collected a new dataset of people referring to one of four objects in a tabletop setting and demonstrate that our approach is able to infer the correct object with 90% accuracy. Additionally, we augment and improve our filter in a simulated home kitchen domain by learning contextual knowledge in an unsupervised manner from existing written text, increasing our maximum accuracy to 96%, even with an increase in the number of objects from four to seventy.

show abstract

Comparing Heads-up, Hands-free Operation of Ground Robots to Teleoperation

Cited by 7 publications

References 36 publications

Person-following by autonomous robots: A categorical overview

Person-following by autonomous robots: A categorical overview

Miscommunication Detection and Recovery in Situated Human–Robot Dialogue

Interpreting multimodal referring expressions in real time

Contact Info

Product

Resources

About