A Factor Graph Approach to Multi-camera Extrinsic Calibration on Legged Robots

Reinke, Andrzej; Camurri, Marco; Semini, Claudio

doi:10.1109/irc.2019.00071

Cited by 5 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The frames located at the sensor origins are also rigidly attached to the floating base, namely: the camera optical frame C , the IMU frame I , and the LIDAR frame L . The relative locations of these frames are known by design or can be retrieved with calibration procedures, such as the ones described in Furgale et al ( 2013 ) and Reinke et al ( 2019 ). One or more temporal contact frames K are created when a foot comes into contact with the ground.…”

Section: Problem Statementmentioning

confidence: 99%

Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper, we present a modular and flexible state estimation framework for legged robots operating in real-world scenarios, where environmental conditions, such as occlusions, low light, rough terrain, and dynamic obstacles can severely impair estimation performance. At the core of the proposed estimation system, called Pronto, is an Extended Kalman Filter (EKF) that fuses IMU and Leg Odometry sensing for pose and velocity estimation. We also show how Pronto can integrate pose corrections from visual and LIDAR and odometry to correct pose drift in a loosely coupled manner. This allows it to have a real-time proprioceptive estimation thread running at high frequency (250–1,000 Hz) for use in the control loop while taking advantage of occasional (and often delayed) low frequency (1–15 Hz) updates from exteroceptive sources, such as cameras and LIDARs. To demonstrate the robustness and versatility of the approach, we have tested it on a variety of legged platforms, including two humanoid robots (the Boston Dynamics Atlas and NASA Valkyrie) and two dynamic quadruped robots (IIT HyQ and ANYbotics ANYmal) for more than 2 h of total runtime and 1.37 km of distance traveled. The tests were conducted in a number of different field scenarios under the conditions described above. The algorithms presented in this paper are made available to the research community as open-source ROS packages.

show abstract

Section: Problem Statementmentioning

confidence: 99%

Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…By fixing calibration boards on the feet, Blöchliger et al [6] proposed an automatic extrinsic calibration approach by formulating a non-linear least squares problem using visual measurements and estimated 33 unknown parameters. With a similar mechanical setup, Reinke et al [7] proposed a factor graph-based approach which jointly optimized the mutual position of cameras and the robot base using kinematics and fiducial markers. However, the above approaches made significant mechanical modifications and motion of the quadruped robot is even more restricted.…”

Section: Related Workmentioning

confidence: 99%

A Two-Stage Perception System Calibration Algorithm for Quadruped Robots Using Factor Graph

Sun,

Wu,

Liu

et al. 2023

Advances in Transdisciplinary Engineering

View full text Add to dashboard Cite

The accurate calibration of the perception system is one of the key prerequisites for quadruped robots to move safely and robustly in challenging environments. Not only does it provide relative transformations of the sensors w.r.t the robot body but also the time offset between unsynchronized sensors. This paper presents a two-stage spatio-temporal calibration framework for the body-eye calibration of quadruped robots. By attaching one complementary sensor, we first estimate the time offset between the sensor frame and body frame without complicated modifications to the mechanical system. Then we estimate the extrinsic transformation by formulating a hand-eye calibration problem while using the information from the kinematic model and encoder measurements. The extrinsic parameters are finally optimized by minimizing the re-projection error and the kinematics error in a factor graph formulation with a Lie group state representation. The experiments demonstrate that the proposed algorithm produces precise calibration results which guarantee high-accuracy environmental mapping using an RGB-D camera.

show abstract

“…Reinke et al [29] propose an offline method for finding the relative poses between a set of (two) cameras and the base frame of a quadruped robot. They use a fiducial marker mounted on the end-effector of a limb as the calibration target.…”

Section: Related Workmentioning

confidence: 99%

Online Marker-Free Extrinsic Camera Calibration Using Person Keypoint Detections

Bastian

Bultmann

Behnke

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Calibration of multi-camera systems, i.e. determining the relative poses between the cameras, is a prerequisite for many tasks in computer vision and robotics. Camera calibration is typically achieved using offline methods that use checkerboard calibration targets. These methods, however, often are cumbersome and lengthy, considering that a new calibration is required each time any camera pose changes. In this work, we propose a novel, marker-free online method for the extrinsic calibration of multiple smart edge sensors, relying solely on 2D human keypoint detections that are computed locally on the sensor boards from RGB camera images. Our method assumes the intrinsic camera parameters to be known and requires priming with a rough initial estimate of the camera poses. The person keypoint detections from multiple views are received at a central backend where they are synchronized, filtered, and assigned to person hypotheses. We use these person hypotheses to repeatedly solve optimization problems in the form of factor graphs. Given suitable observations of one or multiple persons traversing the scene, the estimated camera poses converge towards a coherent extrinsic calibration within a few minutes. We evaluate our approach in real-world settings and show that the calibration with our method achieves lower reprojection errors compared to a reference calibration generated by an offline method using a traditional calibration target.

show abstract

A Factor Graph Approach to Multi-camera Extrinsic Calibration on Legged Robots

Cited by 5 publications

References 15 publications

Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios

Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios

A Two-Stage Perception System Calibration Algorithm for Quadruped Robots Using Factor Graph

Online Marker-Free Extrinsic Camera Calibration Using Person Keypoint Detections

Contact Info

Product

Resources

About