Buttoning Task with a Dual-Arm Robot: An Exploratory Study on a Marker-based Algorithmic Method and Marker-less Machine Learning Methods

Fujii, Wakana; Suzuki, Kanata; Ando, Tomoki; Tateishi, Ai; Mori, Hiroki; Ogata, Tetsuya

doi:10.1109/sii52469.2022.9708612

Cited by 4 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In that case, we will evaluate the actual planning time and task completion time. In addition, we will work toward a fusion with our imitation learning models for robot motion [10][11][12][13][14] .…”

Section: Discussionmentioning

confidence: 99%

“…Offline task planning is important in long-horizon robot tasks. The more complex the task, the more important it is to have a clear vision before taking action 12 . Research into offline robot task planning using an LLM is being conducted to output task instructions by humans as multiple action sequences.…”

Section: Offline Task Planning Using An Llmmentioning

confidence: 99%

“…In the imitation learning framework, it can acquire a model directly from the robot experience if effective data for task execution are available. Our previous studies also showed that the imitation learning method can perform flexible object manipulations such as clothing 10,11 buttoning 12 , in-air knotting 13 , and cooking scrambled eggs 14 . In addition, many efforts are being made to link simple language instruction with imitation learning [15][16][17][18] .…”

Section: Introductionmentioning

confidence: 96%

See 2 more Smart Citations

Enhancement of Long-Horizon Task Planning via Active and Passive Modification in Large Language Model

Hori,

Suzuki,

Ogata

2024

Preprint

View full text Add to dashboard Cite

This study proposes a method for generating complex, long-horizon off-line task plans using large language models (LLMs). Although several studies have been conducted in recent years on robot task planning using LLMs, the planning results are often simple. In the proposed method, the LLM actively collects missing information by asking questions, and the task plan is upgraded with a one-shot dialog example. In addition, the robustness of the task planning is enhanced by using the method that modifies the plan results based on human instructions. The effectiveness of the proposed method is demonstrated through dialogue experiments using a cooking task as the subject. Furthermore, the proposed method can be applied to instructions that include images.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Offline Task Planning Using An Llmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

See 1 more Smart Citation

Enhancement of Long-Horizon Task Planning via Active and Passive Modification in Large Language Model

Hori,

Suzuki,

Ogata

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…We have since extended the deep predictive learning model to realize a knot-tying [32] and button-hanging [33] tasks with the same model. We confirmed that including detailed object information such as proximity and tactile sensors as model inputs improved prediction performance [32], [34].…”

Section: Examples Of Various Taskmentioning

confidence: 99%

Composition of Robot Motions based on the Concept of Deep Predictive Learning

Suzuki

Ito

Yamada

et al. 2022

JRSJ

Self Cite

View full text Add to dashboard Cite

A deep learning-based approach can generalize model performance while reducing feature design costs by learning end-to-end environment recognition and motion generation. However, the process incurs huge training data collection costs and time and human resources for trial-and-error when involving physical contact with robots. Therefore, we propose "deep predictive learning," a motion learning concept that assumes imperfections in the predictive model and minimizes the prediction error with the real-world situation. Deep predictive learning is inspired by the "free energy principle and predictive coding theory," which explains how living organisms behave to minimize the prediction error between the real world and the brain. Robots predict near-future situations based on sensorimotor information and generate motions that minimize the gap with reality. The robot can flexibly perform tasks in unlearned situations by adjusting its motion in realtime while considering the gap between learning and reality. This paper describes the concept of deep predictive learning, its implementation, and examples of its application to real robots. The code and document are available at https: //ogata-lab.github.io/eipl-docs

show abstract

Sensorimotor Attention and Language-based Regressions in Shared Latent Variables for Integrating Robot Motion Learning and LLM

Suzuki,

Ogata

2024

2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Buttoning Task with a Dual-Arm Robot: An Exploratory Study on a Marker-based Algorithmic Method and Marker-less Machine Learning Methods

Cited by 4 publications

References 17 publications

Enhancement of Long-Horizon Task Planning via Active and Passive Modification in Large Language Model

Enhancement of Long-Horizon Task Planning via Active and Passive Modification in Large Language Model

Composition of Robot Motions based on the Concept of Deep Predictive Learning

Sensorimotor Attention and Language-based Regressions in Shared Latent Variables for Integrating Robot Motion Learning and LLM

Contact Info

Product

Resources

About