Controlling Industrial Robots with High-Level Verbal Commands

Choi, Dongkyu; Wei, Shi; Liang, Ying; Yeo, Kheng Hui; Kim, Jung‐Jae

doi:10.1007/978-3-030-90525-5_19

Cited by 3 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 Related works At the present time, there are no publications in the literature on the topic of controlling a robotic platform using Russian commands on natural language. At the same time, there are a number of works in English (Gubbi, Upadrashta and Amrutur, 2020;Min et al, 2021;Ahn et al, 2022;Choi et al, 2021) related to the control of robotic platform using language models of neural network.…”

Section: Introductionmentioning

confidence: 99%

“…The authors of the paper (Choi et al, 2021) propose an adaptation based on language models GPT-2 (Radford et al, 2019) and T5 (Raffel, Shazeer and Roberts, 2019) with transformer topology for processing natural language instructions when controlling industrial robotic devices. In work, the stage of validation of commands coming from a human (feedback) based on the ICARUS cognitive architecture (Choi and Langley, 2018) is proposed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A deep learning method based on language models for processing natural language Russian commands in human robot interaction

2023

RR. Т. and A.L.

View full text Add to dashboard Cite

The development of high performance human-machine interface systems for controlling robotic platforms by natural language is a relevant task in interdisciplinary field «Human-Robot Interaction». In particular, it is in demand, when the robotic platform is controlled by an operator without any skills necessary to use specialized control tools. The paper describes a complex Russian language commands processing into a formalized RDF graph format to control a robotic platform. In this processing, neural network models are consistently used to search and replace pronouns in commands, restore missing verbs-actions, decompose a complex command with several actions into simple commands with only one action and classify simple command attribute. State-of-the-art solutions are applied as neural network models in this work. It is language models based on deep neural networks transformer architecture. The previous our papers show synthetic datasets based on developed generator of Russian language text commands, data based on crowdsourcing technologies and data from open sources for each of the described stages of processing. These datasets were used to fine-tune the language models of the neural networks. In this work, the resulting fine-tuned language models are implemented into the interface. The impact of the stage of searching and replacing pronouns on the efficiency of command conversion are evaluated. Using the virtual three-dimensional robotic platform simulator created at the National Research Center «Kurchatov Institute», the high efficiency of complex Russian language commands processing as part of a human-machine interface system is demonstrated.

show abstract

Section: Introductionmentioning

confidence: 99%