Force based manipulation of Jenga blocks

Kimura, Shinya; Watanabe, Tsutomu; Aiyama, Yasumichi

doi:10.1109/iros.2010.5651753

Cited by 8 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A fine-grained kinematic analysis of the physics behind weight, friction interactions, and stability of the Jenga tower is studied in [ 28 ], both during and after the extraction. With a 6DOF manipulator, a custom gripper, and a 6-axis force sensor, they compare the real forces with the modeled ones and achieve 14 consecutive extractions before breaking the tower.…”

Section: Related Workmentioning

confidence: 99%

Deep Instance Segmentation and Visual Servoing to Play Jenga with a Cost-Effective Robotic System

Marchionna

Pugliese

Martini

et al. 2023

Sensors

View full text Add to dashboard Cite

The game of Jenga is a benchmark used for developing innovative manipulation solutions for complex tasks. Indeed, it encourages the study of novel robotics methods to successfully extract blocks from a tower. A Jenga game involves many traits of complex industrial and surgical manipulation tasks, requiring a multi-step strategy, the combination of visual and tactile data, and the highly precise motion of a robotic arm to perform a single block extraction. In this work, we propose a novel, cost-effective architecture for playing Jenga with e.Do, a 6DOF anthropomorphic manipulator manufactured by Comau, a standard depth camera, and an inexpensive monodirectional force sensor. Our solution focuses on a visual-based control strategy to accurately align the end-effector with the desired block, enabling block extraction by pushing. To this aim, we trained an instance segmentation deep learning model on a synthetic custom dataset to segment each piece of the Jenga tower, allowing for visual tracking of the desired block’s pose during the motion of the manipulator. We integrated the visual-based strategy with a 1D force sensor to detect whether the block could be safely removed by identifying a force threshold value. Our experimentation shows that our low-cost solution allows e.DO to precisely reach removable blocks and perform up to 14 consecutive extractions in a row.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Instance Segmentation and Visual Servoing to Play Jenga with a Cost-Effective Robotic System

Marchionna

Pugliese

Martini

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…A different line of research is Kimura et al (2010) where physical force is explicitly formulated with respect to the tower structure for planning. In our work, we do not do explicit formation of contact force as in Kimura et al (2010), nor do we perform trials on-site for evaluating the robot's operation. We only use physics engine to acquire synthesized data for training the visual-physics model.…”

Section: Blocks-based Manipulation Tasksmentioning

confidence: 99%

Learning Manipulation under Physics Constraints with Visual Perception

Li,

Leonardis,

Bohg

et al. 2019

Preprint

View full text Add to dashboard Cite

Understanding physical phenomena is a key competence that enables humans and animals to act and interact under uncertain perception in previously unseen environments containing novel objects and their configurations. In this work, we consider the problem of autonomous block stacking and explore solutions to learning manipulation under physics constraints with visual perception inherent to the task. Inspired by the intuitive physics in humans, we first present an end-to-end learning-based approach to predict stability directly from appearance, contrasting a more traditional model-based approach with explicit 3D representations and physical simulation. We study the model's behavior together with an accompanied human subject test. It is then integrated into a realworld robotic system to guide the placement of a single wood block into the scene without collapsing existing tower structure. To further automate the process of consecutive blocks stacking, we present an alternative approach where the model learns the physics constraint through the interaction with the environment, bypassing the dedicated physics learning as in the former part of this work. In particular, we are interested in the type of tasks that require the agent to reach a given goal state that may be different for every new trial. Thereby we propose a deep reinforcement learning framework that learns policies for stacking tasks which are parametrized by a target structure.

show abstract

“…A different line of research is [23] where physical force is explicitly formulated with respect to the tower structure for planning. In our work, we do not do explicit formation of contact force as in [23], nor do we perform trials on-site for evaluating the robot's operation. We only use physics engine to acquire synthesized data for training the visualphysics model.…”

Section: Related Workmentioning

confidence: 99%

“…[22] improved on [21] by further incorporating a physics engine to initialize the candidates for pulling test. A different line of research is [23] where physical force is explicitly formulated with respect to the tower structure for planning. In our work, we do not do explicit formation of contact force as in [23], nor do we perform trials on-site for evaluating the robot's operation.…”

Section: Related Workmentioning

confidence: 99%

Visual stability prediction for robotic manipulation

Leonardis

Fritz

2017

2017 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Understanding physical phenomena is a key competence that enables humans and animals to act and interact under uncertain perception in previously unseen environments containing novel objects and their configurations. Developmental psychology has shown that such skills are acquired by infants from observations at a very early stage.In this paper, we contrast a more traditional approach of taking a model-based route with explicit 3D representations and physical simulation by an end-to-end approach that directly predicts stability from appearance. We ask the question if and to what extent and quality such a skill can directly be acquired in a data-driven way-bypassing the need for an explicit simulation at run-time.We present a learning-based approach based on simulated data that predicts stability of towers comprised of wooden blocks under different conditions and quantities related to the potential fall of the towers. We first evaluate the approach on synthetic data and compared the results to human judgments on the same stimuli. Further, we extend this approach to reason about future states of such towers that in turn enables successful stacking.

show abstract

Force based manipulation of Jenga blocks

Cited by 8 publications

References 7 publications

Deep Instance Segmentation and Visual Servoing to Play Jenga with a Cost-Effective Robotic System

Deep Instance Segmentation and Visual Servoing to Play Jenga with a Cost-Effective Robotic System

Learning Manipulation under Physics Constraints with Visual Perception

Visual stability prediction for robotic manipulation

Contact Info

Product

Resources

About