To effectively cooperate with a human, advanced manufacturing machines are expected to execute the industrial tasks according to human natural language (NL) instructions. However, NL instructions are not explicit enough to be understood and are not complete enough to be executed, leading to incorrected executions or even execution failure. To address these problems for better execution performance, we developed a Natural-Language-Instructed Task Execution (NL-Exe) method. In NL-Exe, semantic analysis is adopted to extract task-related knowledge, based on what human NL instructions are accurately understood. In addition, logic modeling is conducted to search the missing execution-related specifications, with which incomplete human instructions are repaired. By orally instructing a humanoid robot Baxter to perform industrial tasks “drill a hole” and “clean a spot”, we proved that NL-Exe could enable an advanced manufacturing machine to accurately understand human instructions, improving machine’s performance in industrial task execution.

It is the first time that 3-D gaze is utilized in a real environment to command a robot for a practical application. Three-dimensional gaze tracking is promising as an intuitive alternative for human-robot interaction especially for disabled and elderly people who cannot handle the conventional interaction modalities.

No abstract

No abstract

No abstract