tech united 2015/2016 - robocup · • tegra k1 soc – nvidia kepler gpu with 192 cuda cores –...
TRANSCRIPT
![Page 1: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/1.jpg)
Tech United 2015/2016Tech United 2015/2016
Workshop Workshop –– November 2015, Aveiro, PortugalNovember 2015, Aveiro, Portugal
![Page 2: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/2.jpg)
Content
Planning toward RoboCup 2016
•Kinect 2
•Shooting at goal
•Faster gameplay
![Page 3: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/3.jpg)
Content
Planning toward RoboCup 2016
•Kinect 2
•Shooting at goal
•Faster gameplay
![Page 4: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/4.jpg)
Kinect v1 has shortcomings
• Currently using Kinect v1
– CCD has low sensitivity:
– CCD has bad quality colors
– Stability problems
– Depth range limited to 6 m
• Full speed ball (10 m/s) arrives in 0.6 seconds after first
possible detection
Much light Low exposure 30 Hz image stream
Low-light High exposure 15 Hz image stream
![Page 5: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/5.jpg)
Solution: Kinect v2
• Use the Kinect v2
– Higher quality CCD
• More sensitive: always 30 Hz
• Better color quality: easier to find the ball
– Less stability issues (hopefully ☺)
– Increased depth range to 9 m
• Time-of-flight i.s.o structured light
– Increased view angle
![Page 6: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/6.jpg)
Implementation: GPU dev board
• Kinect v2 requires GPU
– Robot pc’s: Beckhoff industrial PC
• No GPU
• No space
• Solution:
– Jetson TK1 dev board
![Page 7: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/7.jpg)
Jetson TK1
• Tegra K1 SOC
– NVIDIA Kepler GPU with 192 CUDA Cores
– NVIDIA 4-Plus-1™ Quad-Core
– ARM® Cortex™-A15 CPU
RS232 Serial PortUSB3.0 GigE LAN
Micro-USB2.0
Half Mini-PCIe Slot
![Page 8: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/8.jpg)
Ball detection
• Registration of depth and color image
• Color segmentation using Google annotated
database
• Floodfill algorithm for blob detection
• Blob selection based on:
– Size:distance ratio
– Width:height ratio
• Transformation to robot coordinates
![Page 9: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/9.jpg)
Jetson implementation
• GPU
– JPEG decompression of color stream
– Depth reconstruction from IR phase images
– Floodfill
• CPU
– Registration (near future: GPU)
– Color segmentation (fast enough)
– Blob selection
![Page 10: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/10.jpg)
Result
• 30 Hz ball detection
• Accuracy still unknown
![Page 11: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/11.jpg)
Future work
• Use field to compensate for mechanical tilt
– Adopt from Kinect v1 implementation
• Iteration with SVD and removal of outliers
• Lowpass filter on tilt angles
• Detect obstacles
– Remove field from registrated image
– Everything left is ball or obstacle
• Detect lines + goal
– Improves sensor fusion + shot accuracy
![Page 12: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/12.jpg)
Communication with Jetson
• Re-use of RTDB/comm
– Second comm process
– Flexible interprocess
communication
• Possible performance /
reliability improvements
– Remote records
– TCPIP i.s.o UDP
– Triggered i.s.o fixed rate
Robot
Robot PC
RTDB
Motion
Vision
Worldmodel
Comm
wifi
Jetson boardComm
Comm
LAN
RTDB
Kinect process
![Page 13: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/13.jpg)
Content
Planning toward RoboCup 2016
•Kinect 2
•Shooting at goal
•Faster gameplay
![Page 14: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/14.jpg)
Shooting at goal
Efficiency semi-final and final RoboCup 2015
•Semi-final 29% successful goal attempts
•Final 7% successful goal attempts
![Page 15: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/15.jpg)
Shooting at goal
Solution for improvements
•Improve lob shots at goal
•Implement shooting with effect at goal
![Page 16: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/16.jpg)
Improve lob shots
Problems with current lob shots
•Inaccurate
•Shooting angle fixed at 45 degrees
•Low velocity in xy-direction
Goal
•Accurate lob shot with adjustable shooting angle
![Page 17: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/17.jpg)
Improve lob shots
Objectives
•Validate ball model for a bouncing ball
•Determination of relation ball trajectory and lever
settings
•Inverse problem; from desired target xyz to initial
inputs K and L
![Page 18: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/18.jpg)
Improve lob shots
Ball model validation
•Parabolic ball trajectory dependent on gravity, ball
velocity v0 shooting angle α0.
![Page 19: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/19.jpg)
Improve lob shots
Ball trajectory for K and L
•Determination of relation ball trajectory and lever
settings (K and L)
![Page 20: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/20.jpg)
Results
Inverse model; desired target xyz
![Page 21: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/21.jpg)
Implement shooting with effect
![Page 22: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/22.jpg)
Shooting with effect
![Page 23: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/23.jpg)
Shooting with effect
![Page 24: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/24.jpg)
Content
Planning toward RoboCup 2016
•Kinect 2
•Shooting at goal
•Faster gameplay
![Page 25: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/25.jpg)
Faster gameplay
Current strategy implementation
•Role-based strategy
•Fixed behavior implemented in role
•Behavior change requires role switch
Limitations
•Tasks/action coupled to role
•“State” is not preserved during role switching
![Page 26: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/26.jpg)
Faster gameplay
New implementation
•No behavior implemented in roles
•All actions can be shared between roles
•Sequences of actions
Improvement enables
•Faster gameplay
•Easier passing with multiple robots
•Longer planning horizon
![Page 27: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/27.jpg)
www.robocupeuropeanopen.org
![Page 28: Tech United 2015/2016 - RoboCup · • Tegra K1 SOC – NVIDIA Kepler GPU with 192 CUDA Cores – NVIDIA 4-Plus-1™ Quad-Core – ARM® Cortex™-A15 CPU RS232 Serial Port USB3.0](https://reader030.vdocuments.mx/reader030/viewer/2022041104/5f0423817e708231d40c80fa/html5/thumbnails/28.jpg)
Thank you