Integrating multi-modal interfaces to command UAVs

Monajjemi, Valiallah; Pourmehr, Shokoofeh; Sadat, Seyed Abbas; Zhan, Fei; Wawerla, Jens; Mori, Greg

doi:10.1145/2559636.2559646

Cited by 5 publications

(4 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…On the other hand, multimodal commands offer even more possibilities than multimodal displays. The idea is to combine voice, touch and gestures to reach simple, natural and fast interactions between operators and robots [ 24 ]. There are multiple approaches to command multi-robot missions by means of speech commands [ 25 ].…”

Section: State Of Artmentioning

confidence: 99%

Multi-Robot Interfaces and Operator Situational Awareness: Study of the Impact of Immersion and Prediction

Roldán

Peña-Tapia

Martín-Barrio

et al. 2017

Sensors

View full text Add to dashboard Cite

Multi-robot missions are a challenge for operators in terms of workload and situational awareness. These operators have to receive data from the robots, extract information, understand the situation properly, make decisions, generate the adequate commands, and send them to the robots. The consequences of excessive workload and lack of awareness can vary from inefficiencies to accidents. This work focuses on the study of future operator interfaces of multi-robot systems, taking into account relevant issues such as multimodal interactions, immersive devices, predictive capabilities and adaptive displays. Specifically, four interfaces have been designed and developed: a conventional, a predictive conventional, a virtual reality and a predictive virtual reality interface. The four interfaces have been validated by the performance of twenty-four operators that supervised eight multi-robot missions of fire surveillance and extinguishing. The results of the workload and situational awareness tests show that virtual reality improves the situational awareness without increasing the workload of operators, whereas the effects of predictive components are not significant and depend on their implementation.

show abstract

Section: State Of Artmentioning

confidence: 99%

Multi-Robot Interfaces and Operator Situational Awareness: Study of the Impact of Immersion and Prediction

Roldán

Peña-Tapia

Martín-Barrio

et al. 2017

Sensors

View full text Add to dashboard Cite

show abstract

“…Las interacciones multimodales ofrecen aún más posibilidades para los comandos que para la información. La fusión de voz, tacto y gestos hace que las interacciones entre los operadores y los robots sean más fáciles, rápidas y naturales (Monajjemi et al, 2014). Los comandos de voz se han aplicado satisfactoriamente en misiones multi-robot (Kavitha et al, 2015), al igual que los comandos por gestos, usando tanto las manos (Mantecón et al, 2014) como la cara (Nagi et al, 2014).…”

Section: Interacciones Multimodalesunclassified

Una revisión de los sistemas multi-robot: desafíos actuales para los operadores y nuevos desarrollos de interfaces

Roldán‐Gómez

Rivas

Garcia-Aunon

et al. 2020

Rev. iberoam. autom. inform. ind.

View full text Add to dashboard Cite

<p class="icsmabstract">Los sistemas multi-robot están experimentando un gran desarrollo en los últimos tiempos, ya que mejoran el rendimiento de las misiones actuales y permiten realizar nuevos tipos de misiones. Este artículo analiza el estado del arte de los sistemas multi-robot, abordando un conjunto de temas relevantes: misiones, flotas, operadores, interacción humano-sistema e interfaces. La revisión se centra en los retos relacionados con factores humanos como la carga de trabajo o la conciencia de la situación, así como en las propuestas de interfaces adaptativas e inmersivas para solucionarlos.</p>

show abstract

“…In 2013, Monajjemi et al presented a method to command a team of UAVs by using face and hand gestures [71]. Later, Monajjemi et al extended their work by commanding a team of two UAVs using not only face engagement and hand gestures but also voice and touch interfaces [70]. Similar to Monajjemi's works on multi-modal interaction above, MohaimenianPour & Vaughan [69] and Nagi et al [75] realized UAV control with hands and faces by relying on visual object detectors and simple preset rules Unlike Monajjemi's works on multi-modal interaction above, Sun et al focused on piloting a drone with gesture recognition by combining a visual tracker with a skin pixel detector for robust performance [102].…”

Section: User Interfaces For Uavsmentioning

confidence: 99%

“…Human-accompanying model should integrate functions of human approaching, following, leading, side-by-side walking, bird-eye viewing on the top for a more natural HDI. Similarly, human-sensing interface should integrate at least four modalities of human tracking, hand tracking, face tracking, and voice interaction (e.g., [70]). In the engineering field, UAVs should also perform environment sensing at the same time so that they can accompany their users without hitting obstacle (e.g., [65]).…”

Section: Guidelines and Recommendationsmentioning

confidence: 99%

Companion Unmanned Aerial Vehicles: A Survey

Liew¹,

Yairi²

2020

Preprint

View full text Add to dashboard Cite

Recent technological advancements in smallscale unmanned aerial vehicles (UAVs) have led to the development of companion UAVs. Similar to conventional companion robots, companion UAVs have the potential to assist us in our daily lives and to help alleviating social loneliness issue. In contrast to ground companion robots, companion UAVs have the capability to fly and possess unique interaction characteristics. Our goals in this work are to have a bird's-eye view of the companion UAV works and to identify lessons learned and guidelines for the design of companion UAVs. We tackle two major challenges towards these goals, where we first use a coordinated way to gather topquality human-drone interaction (HDI) papers from three sources, and then propose to use a perceptual map of UAVs to summarize current research efforts in HDI. While being simple, the proposed perceptual map can cover the efforts have been made to realize companion UAVs in a comprehensive manner and lead our discussion coherently. We also discuss patterns we noticed in the literature and some lessons learned throughout the review. In addition, we recommend several areas that are worth exploring and suggest a few guidelines to enhance HDI researches with companion UAVs.

show abstract

Integrating multi-modal interfaces to command UAVs

Cited by 5 publications

References 5 publications

Multi-Robot Interfaces and Operator Situational Awareness: Study of the Impact of Immersion and Prediction

Multi-Robot Interfaces and Operator Situational Awareness: Study of the Impact of Immersion and Prediction

Una revisión de los sistemas multi-robot: desafíos actuales para los operadores y nuevos desarrollos de interfaces

Companion Unmanned Aerial Vehicles: A Survey

Contact Info

Product

Resources

About