3d vision guided robotic charging station for electric and plug-in hybrid vehicles
TRANSCRIPT
3D Vision Guided Robotic Charging Station
for Electric and Plug-in Hybrid Vehicles
J. Mišeikis, M. Rüther, B. Walzel, M. Hirz, H. Brunner
Electric Vehicles are becoming great!
… but EV owners still have plenty to
complain about!
Electric Vehicles are becoming great!
… but EV owners still have plenty to
complain about!
Electric Vehicles are becoming great!
Charging
Image source: http://www.electriccarpledge.com/electric-vehicle-resources/electric-car-plug-types/
Image source: http://www.quick220.com/page/cp-product-solutions-2.html
Image source: https://twitter.com/dunneautomotive/status/780996480417034241
Image source: https://twitter.com/dunneautomotive/status/780996480417034241
Image source: https://twitter.com/dunneautomotive/status/780996480417034241
But the charging cable becomes very thick and heavy!
Let’s allow the robot to
do the dirty work!
Use Case
The car is parked in the designated parking spot
and the robot handles the whole charging
process
Related Work
Our Concept: Robotic EV / PHEV Charging Station
Our System Setup
Algorithm
Detection of the
charging port
3D Pose Estimation
Robot Motion
Planning
3-Step Plug-In
Procedure
Charging!
Algorithm
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Step-by-step
Shape-Based Template Matching: Templates
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Shape-Based Template Matching: Evaluation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Shape-Based Template Matching: Process
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Left Cam Right Cam
Shape-Based Template Matching: Process
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Left Cam Right Cam
Shape-Based Template Matching: Process
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Left Cam Right Cam
Left Cam Right Cam
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Point 1 (X, Y)
Point 2 (X, Y)
Point 3 (X, Y)
Point 1 (X, Y)
Point 2 (X, Y)
Point 3 (X, Y)
Left Cam Right Cam
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
We use stereo triangulation to calculate depth
Left Cam Right Cam
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Real-World Coordinates
Point 1 (X, Y, Z)
Point 2 (X, Y, Z)
Point 3 (X, Y, Z)
Left Cam Right Cam
3D Pose Estimation: Find the Relative Orientation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Roll
Pitch
Yaw
3D Pose Estimation: Find the Relative Orientation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Roll
Pitch
Yaw
We can think about this as 3D plane fitting on selected points
3D Pose Estimation: Find the Relative Orientation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Roll
Pitch
Yaw
We can think about this as 3D plane fitting on selected points
3D Pose Estimation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Charging Port Position in
Real-World Coordinates
(X, Y, Z, roll, pitch, yaw)
Left Cam Right Cam
Marker-less Eye-to-Hand Calibration
Cam to Plug
Transform
End-Effector to
Plug
Transform
Marker-less Eye-to-Hand Calibration
3D Pose Estimation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Now we have the following information:
● Distance to the port from the vision sensor in X, Y and Z
● Orientation of the port in relation to the vision sensor
● Location of the vision sensor in relation to the robot
● Location of the charging plug in relation to the robot
3D Pose Estimation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Let’s move the robot!
Now we have the following information:
● Distance to the port from the vision sensor in X, Y and Z
● Orientation of the port in relation to the vision sensor
● Location of the vision sensor in relation to the robot
● Location of the charging plug in relation to the robot
Robot Motion Planning: RRT-Connect
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Plug-In Procedure: 3 Steps
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Charging! And Unplugging Afterwards
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Simply track-back the same trajectory!
Results
Exp Charging Port at 10° angle Charging Port at 30° angle
1 Success Success: Misalignment
2 Success: Misalignment Failed: Missed rotation
3 Success Success
4 Failed: Missed rotation Success: Misalignment
5 Success: Misalignment Success: Misalignment
Attempts: 10
Successful Charging: 8
Small alignment issues: 5
Failed connection: 2
Due to initial calibration issues
Future Work
● Test on a real vehicle
● Automatically open charging port lid
● Add more safety features
● Automatic communication between
the charger and the vehicle
● Incorporate automatic parking
Thank You!
Any Questions?