common sensing techniques for reactive robots (12-11-7) sungmin lee ( 이성민 ) division of...
TRANSCRIPT
![Page 1: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/1.jpg)
Common Sensing Techniques for Reactive
Robots(12-11-7)
Sungmin Lee ( 이성민 )Division of Electronic Engineering, Chonbuk National
University
Intelligent Systems & Robotics Lab.http://robotics.jbnu.ac.kr
![Page 2: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/2.jpg)
Chapter objectives
• Describe difference between active and passive sensors
• Describe the types of behavioral sensor fusion
• Define each of the following terms in one or two sentences: proprioception, exteroception, exproprioception,proximity sensor, logical sensor, false positive, false negative, hue, saturation, computer vision
• Describe the problems of specular reflection, cross talk, foreshortening, and if given a 2D line drawing of surfaces, illustrate where each of these problems would be likely to occur
• If given a small interleaved RGB image and a range of color values for a region, be able to 1) threshold on color and 2) construct a color histogram
![Page 3: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/3.jpg)
Contents
1. Logical sensors
2. Behavioral Sensor Fusion
3. Attributes of a sensor
4. Sensor Categories
5. Computer vision
6. Case study
7. Summary
![Page 4: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/4.jpg)
Motivation
• Sensing is tightly coupled with acting in reactive systems, so need to know about sensors
• What sensors are out there?– Ultrasonics, cameras are traditional favorites– Sick laser ranger is gaining fast in popularity
• How would you describe them (attributes)?
• How would you decide which one to pick and use for an application?
![Page 5: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/5.jpg)
Logical Sensors
• A unit of sensing or module (supplies a particular percept).
• It consists of the signal processing and the software processing.
• Can be easily implemented as a perceptual schema.
• Different sensors/perceptual schemas can produce the same percept - motor schema doesn’t care!– Behavior can pick what’s available
• Example: ring of IRs, ring of sonars– If sensor fails, then another can be substituted
without deliberation or explicit modeling– Conflicts in allocation can be solved by using
logical sensors (deliberation is required to assign)
![Page 6: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/6.jpg)
Active vs. Passive (Example)
• Active sensors- Sensor emits some form of energy and then measures the impact as a way of understanding the environment- Ex. Ultrasonics, laser
• Passive sensors- Sensor receives energy already in the environment- Ex. Camera
• Passive consume less energy, but often signal-noise problems
• Active often have restricted environments
StereoCamera
pair
Thermalsensor
Laserranger
Sonars
Bumpsensor
![Page 7: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/7.jpg)
Behavioral Sensor Fusion
Sensor fusion is a broad term used for any process that combines information from multiple sensors into a single percept.In some cases multiple sensors are used when a particular sensor is too imprecise or noisy to give reliable data. Adding a second sensor can give another “vote” for the percept.
When a sensor leads the robot to believe that a percept is present, but it is not, the error is called a false positive.
The robot has made a positive identification of percept, but it was false. Likewise, an error where the robot misses a percept is known as a false negative.
False positive
False negative
![Page 8: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/8.jpg)
Sensing Model
11
Sensor/Transducer
Behavior
Action
![Page 9: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/9.jpg)
Sensing in Reactive Paradigm
Each behavior has its own dedicated sensing. One behavior literally does not know what another behavior is doing or perceiving.
Behavior
Behavior
Behavior
![Page 10: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/10.jpg)
PerceptualSchemas
Motor Schemas
Behavioral Sensor Fusion: -sensor fission
This sensor fission in part as a take off on the connotations of the word “fusion” in nuclear physics. In nuclear fusion, energy is created by forcing atoms and particles together, while in fission, energy is creating by separating atoms and particles.
![Page 11: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/11.jpg)
PerceptualSchema
Motor Schema
Behavioral Sensor Fusion: -action-oriented sensor fusion
This type of sensor fusion is called action-oriented sensor fusion to emphasize that the sensor data is being transformed into a behavior-specific representation in order to support a particular action, not for constructing a world model.
![Page 12: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/12.jpg)
PerceptualSchema
Motor Schema
Behavioral Sensor Fusion: -sensor fashion
Sensor fashion, an alliterative name intended to imply the robot was changing sensors with changing circumstances just as people change styles of clothes with the seasons.
![Page 13: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/13.jpg)
Designing a Sensor Suite-Attributes of a sensor
•Field of view, range : does it cover the “right” area
•Accuracy & repeatability : how well does it work?
•Responsiveness in target domain : how well does it work for
this domain?
•Power consumption : may suck the batteries dry too fast
•Reliability : can be a bit flakey, vulnerable
•Size : always a concern!
•Computational Complexity : can you process it fast enough?
•Interpretation Reliability : do you believe what it’s saying?
![Page 14: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/14.jpg)
Should be considered for the entire sensing suite :•Simplicity•Modularity•Redundancy - physical redundancy (there are several instances of physically identical sensors on the robot.)
-logical redundancy(another sensor using a different sensing modality
can produce the same percept or releaser.)- fault tolerance
Designing a Sensor Suite-Attributes of a sensor suite
![Page 15: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/15.jpg)
Sensor Categories
• Proprioceptive– Inertial Navigation System(INS)– Global Positioning System(GPS)
• Exteroceptive– Proximity
• Range• Contact
– Computer Vision
![Page 16: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/16.jpg)
Proprioceptive Sensors(1)-Inertial navigation system (INS)
Measure movements electronically through miniature accelerometers INS can provide accurate dead reckoning to 0.1 percent of the distance traveled.However, this technology is unsuitable for mobile robots for several reasons.(cost, Size, etc)
MQ-9 Reaper
![Page 17: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/17.jpg)
GPS systems work by receiving signals from satellites orbiting the Earth.GPS is not complete solutions to the dead reckoning problem in mobile robots.GPS does not work indoors(environmental limit)
Proprioceptive Sensors(2)-Global Positioning System (GPS)
![Page 18: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/18.jpg)
Proximity Sensors(1)-Sonar or ultrasonic
•Sonar refers to any system for using sound to measure range. (use a sonar for underwater vehicles ).
•Ground vehicles commonly use sonar with an ultrasonic frequency.
•Ultrasonic sensors generate high frequency sound waves and evaluate the echo which is received back by the sensor. Sensors calculate the time interval between sending the signal and receiving the echo to determine the distance to an object.
•Ultrasonic is possibly the most common sensor on commercial robots operating
Polaroid ultrasonic transducer
![Page 19: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/19.jpg)
chairs, tables– legs, edges too thin for resolution
Proximity Sensors(1)- Three problems with sonar range readings
foreshortening
cross-talk
specular reflection
![Page 20: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/20.jpg)
Maps produced by a mobile robot using sonars in: a.) a lab and b.) a hallway. (The black line is the path of the robot.)
Proximity Sensors(1)- sonar maps
lab hallway
![Page 21: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/21.jpg)
• Power consumption– High
• Reliability– Lots of problems
• Size– Size of a Half dollar, board is similar size and can be
creatively packaged• Computational Complexity
– Low; doesn’t give much information• Interpretation Reliability
– poor
Proximity Sensors(1)- Attributes of ultrasonic
![Page 22: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/22.jpg)
• Physics : active sensor, works on time of flight
• Advantages : range, inexpensive ($30 US), small
• Disadvantages : specular reflection, crosstalk, foreshortening, high power consumption, low resolution
Proximity Sensors(1)- Ultrasonic Summary
![Page 23: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/23.jpg)
They emit near-infrared energy and measure whether any significant amount of the IR light is returned. These often fail in practice because the light emitted is often “washed out” by bright ambient lighting or is absorbed by dark materials (i.e., the environment has too much noise).
Proximity Sensors(2)- Infrared ray (IR)
Sharp GP2Y0A21YK
![Page 24: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/24.jpg)
Popular class of robotic sensing is tactile, or touch, done with bump and feeler sensors. The sensitivity of a bump sensor can be adjusted for different contact pressures
Proximity Sensors(3)- Bump and feeler sensors
Roomba 500
Bump
![Page 25: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/25.jpg)
Computer Vision- Definition
Computer vision refers to processing data from any modality which uses the electromagnetic spectrum which produces an image.
face recognition
![Page 26: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/26.jpg)
Computer Vision- Attributes
•Physics : light reflecting off of surfaces, respond to wavelenght
•Field of view, range : depends on lens; lens typical have a different VFOV and HFOV (vertical, horizontal)
•Accuracy & repeatability : good
•Responsiveness in target domain : depends on lighting source, inherent constrast between objects of interest
•Power consumption : low
•Reliability : good
•Size : can be miniaturized
•Interpretation Reliability : good
![Page 27: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/27.jpg)
• A charge-coupled device (CCD) is a device for the movement of electrical charge, usually from within the device to an area where the charge can be manipulated, for example conversion into a digital value.
Computer Vision- CCD cameras
• CCD sensors typically produce less NOISE.• CCD sensors typically are more light-sensitive.• CMOS sensors use far less power.• CMOS sensor cost less to produce.
![Page 28: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/28.jpg)
• RGB (red, green, blue) is the NTSC output– Poor color constancy in “real world”
• H,S,I (hue, saturation, intensity) has theoretical color constancy – But not with conversion from RGB to HSI
• Alternatives SCT (Spherical Coordinate Transform) That color space was designed to transform RGB data to a color space that more closely duplicates the response of the human eye.
Computer Vision- Color planes
Original image
RGB
HSI
SCT
![Page 29: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/29.jpg)
• For reactive applications:– Color segmentation
• Imprint on a color region, then follow it (or remember it)
– Color histogramming• Imprint on a region with a distribution of color,
then follow it (or remember it)
Computer Vision- Common Vision Algorithms
![Page 30: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/30.jpg)
Range from Vision-Stereo camera pairs
• Using two cameras to extract range data is often referred to as range from stereo, stereo disparity, binocular vision, or just plain “stereo.” One way to extract depth is to try to superimpose a camera over each eye.
• Each camera finds the same point in each image, turns itself to center that point in the image, then measures the relative angle. The cameras are known as the stereo pair.
Ways of extracting depth from a pair of cameras
![Page 31: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/31.jpg)
Light striping, light stripers or structured light detectors work by projecting a colored line (or stripe), grid, or pattern of dots on the environment. Then a regular vision camera observes how the pattern is distorted in the image.
Range from Vision-Light stripers
![Page 32: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/32.jpg)
Range from Vision-Laser ranging(Sick)
• Accuracy & repeatability - Excellent results
• Responsiveness in target domain• Power consumption• - High; reduce battery run time by half• Reliability - good• Size - A bit large• Computational Complexity
– Not bad until try to “stack up”• Interpretation Reliability
– Much better than any other ranger
flat surface
an obstacle
a negative obstacle
SICK PLS100
![Page 33: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/33.jpg)
• 180o plane• Advantages: high accuracy, coverage• Disadvantages: 2D, resistant to miniaturization, cost
($13,000 US)
Range from Vision-Laser Ranger Summary
NASA/CMU Nomad robot(Carnegie Mellon University )
![Page 34: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/34.jpg)
Case Study : Hors d’Oeuvres, Anyone? (Borg Shark and Puffer Fish)
Camera pair (redundant):Face color
Laser range:Count treatremoval
Sonars:Avoid obstacles,
If blocked,Puffed up
Digital thermometer:“Face” temperature check
Sensor fusion:ReducedFalse positives,False negativesFrom 27.5% to 0%
![Page 35: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/35.jpg)
State diagrams for the Borg Shark
sonar shaftencoders
map:evidence grid
waypointnavigation:
move to goal,avoid
serving food:finding faces,counting treat
removal
thermal
vision
laser range
sonar
awaitingrefill:
finding faces
vision thermal
at waypointOR
food removed
time limitserving exceeded
fooddepleted
fulltray
![Page 36: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/36.jpg)
Summary
• Design of a sensor suite requires careful consideration– Almost all robots will have proprioception, but
exteroception needs to be closely matched to the task and the environment
• Most common exteroceptive sensors on mobile robots are:– Ultrasonics– Computer vision– Laser range
• Color vision can be hard, almost all vision is computationally expensive unless focus on affordances– Borg shark and Puffer fish with color plus heat– Polly and texture
![Page 37: Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649dff5503460f94ae797c/html5/thumbnails/37.jpg)
Thank you