automatic calibration of a robot manipulator and multi 3d camera system
TRANSCRIPT
Automatic Calibration of a Robot Manipulator and Multi 3D Camera System
Justinas Mišeikis, Kyrre Glette, Ole Jakob Elle and Jim Torresen
Another Calibration Paper…
• Standard calibration methods: Zhang and Bouguet
• Calibrate multiple RGB-D cameras at once
• Robot does all the hard work automatically
• Cameras guide the robot movements
Internal Camera Calibration
RGB to IR Camera Calibration
Uncalibrated CalibratedErrorOffset
External: Eye-to-Hand
Motivation
Image from: http://upenncis581finaltiantianhaoxiang.blogspot.jp/2011/12/camera-calibration-with-help-of-matlab.html
Setup
Kinect V2 Kinect V2
Kinect V1UR5
Robot
Calibration Pattern
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
Detect Checkerboard
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
Step 3a
Initialisation Movement
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
Camera Frame
First Stage Second Stage
Initial Checkerboard Position
Camera Frame
First Stage Second Stage
Initial Checkerboard Position
First Stage
Camera Frame
First Stage Second Stage
Initial Checkerboard Position
First Stage
1. Rotate 2. Pan 3. Tilt
Camera Frame
First Stage Second Stage
Initial Checkerboard Position
First Stage
Camera Frame
First Stage Second Stage
Initial Checkerboard Position
First Stage
Camera Frame
First Stage Second Stage
Initial Checkerboard Position
First Stage
Camera Frame
First Stage Second Stage
Initial Checkerboard Position
Second Stage
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
Cam Drivers
Cam Drivers
Cam Drivers
Detect Checkerboard
Calibration Move Planner
Robot Trajectory Planner
Camera Internal Calibration
Eye-to-Hand Calibration
Robot Drivers
Store Calibration
Images
color and depth images
color and depth images
color and depth images
checker board pos
pos in image
coordinatespos in robot coordinates
target pos in robot coordinates
current robot pos
calibration images
calibration images
camera images
robot move completion
The process is repeated for each of the cameras to get the full system calibration
Results: Internal Calibration
Results: Eye-to-Hand Calibration
Conclusions and Future Work✓ Good for quick camera calibration and recalibration
✓ Useful for systems with changing camera setups
✓ Works with ROS supported cameras and robots
‣ Calibrate only for a part of the image area
‣ Calibrate multiple cameras at once covering areas of overlapping field of view
‣ Use ANNs to learn the best positions of calibration pattern
Automatic Calibration of a Robot Manipulator and Multi 3D Camera SystemJustinas Mišeikis, Kyrre Glette, Ole Jakob Elle and Jim Torresen
Thank You!