Dynamic Cloth Manipulation Considering Variable Stiffness and Material Change Using Deep Predictive Model With Parametric Bias

Kawaharazuka, Kento; Miki, Akihiro; Bando, Masahiro; Okada, Kei; Inaba, Masayuki

doi:10.3389/fnbot.2022.890695

Cited by 12 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In ongoing work, we are developing functionality for estimating material properties on the fly during manipulation and using these estimates to improve manipulation trajectories ( Arnold and Yamazaki, 2022 ). Other work in this Research Topic addresses this issue by means of parametric biases ( Kawaharazuka et al, 2022 ).…”

Section: Discussionmentioning

confidence: 99%

Cloth manipulation planning on basis of mesh representations with incomplete domain knowledge and voxel-to-mesh estimation

2023

View full text Add to dashboard Cite

Cloth manipulation is common in both housework and manufacturing. However, robotic cloth manipulation remains challenging, especially for less controlled and open-goal settings. We consider the problem of open-goal planning for robotic cloth manipulation, with focus on the roles of cloth representation and epistemic uncertainty. Core of our system is a neural network trained as a forward model of cloth behaviour under manipulation, with planning performed through backpropagation. We introduce a neural network-based routine for estimating mesh representations from voxel input, and perform planning in mesh format internally. We address the problem of planning with incomplete domain knowledge by introducing an explicit epistemic uncertainty penalty, using prediction divergence between two instances of the forward model network as a proxy of epistemic uncertainty. This allows us to avoid plans with high epistemic uncertainty during planning. Finally, we introduce logic for handling restriction of grasp points to a discrete set of candidates, in order to accommodate graspability constraints imposed by robotic hardware. We evaluate the system’s mesh estimation, prediction, and planning ability on simulated cloth for sequences of one to three manipulations. Comparative experiments confirm that planning on basis of estimated meshes improves accuracy compared to voxel-based planning, and that epistemic uncertainty avoidance improves performance under conditions of incomplete domain knowledge. Planning time cost is a few seconds. We additionally present qualitative results on robot hardware. Our results indicate that representation format and epistemic uncertainty are important factors to consider for open-goal cloth manipulation planning.

show abstract

Section: Discussionmentioning

confidence: 99%

Cloth manipulation planning on basis of mesh representations with incomplete domain knowledge and voxel-to-mesh estimation

2023

View full text Add to dashboard Cite

show abstract

“…8, the cloth image ξ compressed by AutoEncoder, muscle tension f , and muscle length l are included in s, and the target joint angle θ and target body stiffness k ref are included in u. Using the image of an unfolded cloth as the target value, a control input u to achieve this image is obtained by optimization calculation [21]. A series of motions for table setting is shown in Fig.…”

Section: Table Setting Experiments With Dynamic Cloth Manipulationmentioning

confidence: 99%

Hardware Design and Learning-Based Software Architecture of Musculoskeletal Wheeled Robot Musashi-W for Real-World Applications

Kawaharazuka

Miki

Bando

et al. 2022

2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids)

Self Cite

View full text Add to dashboard Cite

Various musculoskeletal humanoids have been developed so far. While these humanoids have the advantage of their flexible and redundant bodies that mimic the human body, they are still far from being applied to real-world tasks. One of the reasons for this is the difficulty of bipedal walking in a flexible body. Thus, we developed a musculoskeletal wheeled robot, Musashi-W, by combining a wheeled base and musculoskeletal upper limbs for real-world applications. Also, we constructed its software system by combining static and dynamic body schema learning, reflex control, and visual recognition. We show that the hardware and software of Musashi-W can make the most of the advantages of the musculoskeletal upper limbs, through several tasks of cleaning by human teaching, carrying a heavy object considering muscle addition, and setting a table through dynamic cloth manipulation with variable stiffness.

show abstract

“…For x in the xy direction, the motion is arbitrarily specified, and u is changed as follows. (11) where N (µ, σ) is the Gaussian noise with mean µ and variance σ, σ {θ,z} is the variance of the data collection for each direction, τ {min,max} are the minimum and maximum values of τ , and f {min,max} denotes the minimum and maximum values of f . By gradually changing u within the minimum and maximum values, the target values of the proportional control can be shifted and various data can be obtained.…”

Section: B Data Collectionmentioning

confidence: 99%

Learning-Based Wiping Behavior of Low-Rigidity Robots Considering Various Surface Materials and Task Definitions

Kawaharazuka

Kanazawa

Okada

et al. 2022

2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids)

Self Cite

View full text Add to dashboard Cite

In recent years, a number of models that learn the relations between vision and language from large datasets have been released. These models perform a variety of tasks, such as answering questions about images, retrieving sentences that best correspond to images, and finding regions in images that correspond to phrases. Although there are some examples, the connection between these pre-trained vision-language models and robotics is still weak. If they are directly connected to robot motions, they lose their versatility due to the embodiment of the robot and the difficulty of data collection, and become inapplicable to a wide range of bodies and situations. Therefore, in this study, we categorize and summarize the methods to utilize the pre-trained vision-language models flexibly and easily in a way that the robot can understand, without directly connecting them to robot motions. We discuss how to use these models for robot motion selection and motion planning without re-training the models. We consider five types of methods to extract information understandable for robots, and show the results of state recognition, object recognition, affordance recognition, relation recognition, and anomaly detection based on the combination of these five methods. We expect that this study will add flexibility and ease-of-use, as well as new applications, to the recognition behavior of existing robots.

show abstract

Dynamic Cloth Manipulation Considering Variable Stiffness and Material Change Using Deep Predictive Model With Parametric Bias

Cited by 12 publications

References 22 publications

Cloth manipulation planning on basis of mesh representations with incomplete domain knowledge and voxel-to-mesh estimation

Cloth manipulation planning on basis of mesh representations with incomplete domain knowledge and voxel-to-mesh estimation

Hardware Design and Learning-Based Software Architecture of Musculoskeletal Wheeled Robot Musashi-W for Real-World Applications

Learning-Based Wiping Behavior of Low-Rigidity Robots Considering Various Surface Materials and Task Definitions

Contact Info

Product

Resources

About