robotic design: frontiers in visual and tactile sensing
TRANSCRIPT
![Page 1: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/1.jpg)
Robotic Design: Frontiers in Visual and Tactile Sensing
![Page 2: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/2.jpg)
Thank You To Our Sponsors
![Page 3: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/3.jpg)
q This webinar will be available afterwards at www.designworldonline.com & via email
q Q&A at the end of the presentation q Hashtag for this webinar: #DWwebinar
Before We Start
![Page 4: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/4.jpg)
Moderator Presenters
Paul Heney Design World
Gerald Loeb Biomed Concepts
SynTouch
Goksel Dedeoglu PercepTonic
![Page 5: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/5.jpg)
Robotic Design: Frontiers in Visual and Tactile Sensing
presented at the
December 4, 2014
Goksel Dedeoglu, Ph.D. Founder, PercepTonic, LLC
![Page 6: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/6.jpg)
Robotics Then…
Moravec, 1980
SRI’s Shakey, 1967 The perception needs of robots have motivated much of the research in
Computer Vision over the past 50 years.
© PercepTonic, LLC
![Page 7: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/7.jpg)
Moravec, 1980
SRI’s Shakey, 1967
thanks to 50 years of consumer electronics
Pelican Imaging
HTC M8
Subaru
XBOX Kinect
Robotics Today
Prox Dynamics
MAST
Brain Corp.
© PercepTonic, LLC
![Page 8: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/8.jpg)
The Promise of Computer Vision Computer Vision has advanced from factory-floor
automation to real-world problem solving in unconstrained environments
Embedded Ever-Ready Everywhere
© PercepTonic, LLC
![Page 9: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/9.jpg)
Computer Vision: Images In, Information Out LE
NS
• Size and shape • 3-‐‑D position & orientation • Identity, object class • Expression, gesture • Location • Motion • Illumination, weather, …
INFORMATION CAMERAS
Computer Vision algorithms analyze images to extract information about the world.
This enables machines that can perceive.
© PercepTonic, LLC
![Page 10: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/10.jpg)
How Does a Robot “See”?
Contextual understanding: Who, What, Where? Real-time operation with visual feedback (do until …) Learning and adaptation
TeraDeep
OMRON
© PercepTonic, LLC
![Page 11: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/11.jpg)
Tools That Help Robots “See” Vision algorithms customized for tracking, segmentation and recognition Power- and cost-optimized processors with expert partitioning between edge and cloud loads Sensors attuned to the particular needs of the computer vision task Intel
OmniVision
© PercepTonic, LLC
![Page 12: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/12.jpg)
Case Study: 3D Depth Perception Example: Subaru’s EyeSight
© PercepTonic, LLC
![Page 13: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/13.jpg)
How Stereo Vision Creates 3D Depth Perception
• Works by triangulation: the closer an object is, the greater its
parallax between the left-right views.
• Stereo is a passive depth sensing method. Active methods include structured light (Kinect) and Time-of-Flight sensors.
left image right image
+ disparity image (inverse depth)
stereo algorithm
output input
© PercepTonic, LLC
![Page 14: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/14.jpg)
Application: Automotive Safety Obstacles detected via stereo depth Stereo Depth/Disparity output
Source: Texas Instruments for more information, see hap://goksel-‐‑dedeoglu.com/3d-‐‑stereo-‐‑vision.html
© PercepTonic, LLC
![Page 15: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/15.jpg)
Challenges in Embedded Vision
Computer Vision has the potential to fuel the next wave of smart and affordable robots
but…
• The algorithms are pixel-intensive and computationally expensive
• The solutions are still evolving –new algorithms every year!
• The processors and sensors are really optimized for multimedia
© PercepTonic, LLC
![Page 16: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/16.jpg)
Embedded Vision Architectures Our best guess for power-efficient vision computation is
heterogeneous and multi-core architectures.
• Mobile SoCs: Qualcomm Snapdragon, NVIDIA Tegra K1
• Established DSPs and FPGAs, Movidius’ Myriad
• Vision IP: CogniVue, videantis, CEVA, adapteva
• Early examples of fixed-function hardware for vision
• Power-efficient server design starting to adopt the same typical block diagram of modern SoCs
© PercepTonic, LLC
![Page 17: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/17.jpg)
The Future of Computer Vision: Embedded, Ever-ready, and Everywhere From proof-‐‑of-‐‑concept prototypes to products
“PC World” unlimited CPU, GPU, …
“Embedded World” select processors & sensors
“Product World” finalized hardware
© PercepTonic, LLC
![Page 18: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/18.jpg)
We Have Come a Long Way…
Moravec, 1980
SRI’s Shakey, 1967 Consumer Electronics is now driving significant investments in computer vision HW and SW,
helping the robotics vision wish-list come true.
© PercepTonic, LLC
![Page 19: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/19.jpg)
References and Resources Case studies and tutorials at the PercepTonic web site
http://www.PercepTonic.com/case-studies.html
The Embedded Vision Alliance (2011-present) http://www.embedded-vision.com
http://embeddedvisionsummit.com
The IEEE Embedded Vision Workshops (2005-present) http://cvisioncentral.com/evw2014/
© PercepTonic, LLC
![Page 20: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/20.jpg)
Thank You! slides will be available at
http://www.PercepTonic.com/slides/DesignWorld
© PercepTonic, LLC
![Page 21: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/21.jpg)
The Future of Machine Touch®
Gerald E. Loeb, M.D., CEO
![Page 22: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/22.jpg)
Touch is essential for dexterity and perception.
Simple tasks become impossible without it.
![Page 23: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/23.jpg)
The Absence of Machine Touch
Precise Expensive Insensate
Rigid Dangerous
![Page 24: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/24.jpg)
Soft
Sensitive Adaptive
• Works in unstructured environs
• Safe near humans
• Performs a variety of tasks
• Does not need retooling
Robots Doing Human Jobs Need Biomimetic Design
![Page 25: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/25.jpg)
BioTac® Biomimetic Tactile Sensors Supple Fingertips
Multi-Modal Sensing
High Bandwidth
Durable
Field Repairable
Scalable
![Page 26: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/26.jpg)
Live LabView GUI Demo
![Page 27: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/27.jpg)
EPFL Lausanne Allegro Hand
Kuka Arm BioTacs
Dexterous Autonomous
Robots
![Page 28: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/28.jpg)
The Today Show: Popular Mechanics Breakthrough Award
Contact Detection Reflex Enables Dexterous Use of a Myoelectric Prosthetic Hand
![Page 29: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/29.jpg)
Blaine Matulevich, Vikram Pandit, Jeremy Fishel
Biomimetic Inhibitory Reflex for
Fragile Grasp
0 1 2 3 4 5 60
2
4
6
8
10
Preamplified Net EMG Signal (V)
Mot
or C
omm
and
(V)
ContactNo Contact
![Page 30: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/30.jpg)
Performance
on cracker task Speed
Accuracy
![Page 31: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/31.jpg)
error
Part Misalignment Failures
Contact Forces
![Page 32: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/32.jpg)
Ejection
Displacement
Part Misalignment Failures
![Page 33: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/33.jpg)
Contact Detection
Solution with Tactile Feedback
![Page 34: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/34.jpg)
NumaTac® “airbags for robots”
![Page 35: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/35.jpg)
Companies Want to Understand Consumer’s Haptic Perception of Their Products
Focus Groups and Sensory Advisory Panels are expensive, slow and prone to “noise”.
Product releases have $Billions at stake.
BioTac texture robot provides objective data about human
dimensions of product “feel”.
![Page 36: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/36.jpg)
Bayesian Action&Perception: Representing the World in the Brain
G.E. Loeb and J.A. Fishel
Frontiers in Neuroscience
vol. 8 (October, 2014)
![Page 37: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/37.jpg)
Tactile Sensing for Humans-in-the-Loop: Telerobots for Hazardous Environments
• Tactile sensing is essential in low visibility environments.
• Fast reflex responses are more useful than conscious perception.
• Government Applications for Dexterous Telerobots: Ø DOD – ordnance
Ø DARPA – drones
Ø NASA – space
Ø NOAA – underwater
Ø NRC – nuclear reactors
![Page 38: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/38.jpg)
“The Sense of Touch”
Jusepe di Ribera
ca. 1615
Norton Simon Gallery, Pasadena, CA
![Page 39: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/39.jpg)
The Standard in Machine Touch® www.SynTouchLLC.com
Willow Garage PR2 Allegro Hand Schunk Dexterous Hand Shadow Hand
HUBO Robot Robotiq Gripper Barrea Hand Kinova JACO Arm
![Page 40: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/40.jpg)
Questions? Paul Heney Design World [email protected] Twitter: @DW_Editor
Goksel Dedeoglu PercepTonic [email protected] Phone: 214.356.4640
Gerald Loeb SynTouch/Biomed Concepts [email protected] Phone: 213.944.2283
![Page 41: Robotic design: Frontiers in visual and tactile sensing](https://reader037.vdocuments.mx/reader037/viewer/2022103117/55a6468e1a28ab3e2e8b4600/html5/thumbnails/41.jpg)
Thank You q This webinar will be available at
designworldonline.com & email
q Tweet with hashtag #DWwebinar
q Connect with Design World
q Discuss this on EngineeringExchange.com