gist: a mobile robotics application of context-based vision in outdoor environment christian siagian...
Post on 21-Dec-2015
214 views
TRANSCRIPT
Gist: A Mobile Robotics Application of Context-Based Vision in Outdoor Environment
Christian SiagianLaurent Itti
Univ. Southern California, CA, USA
Outline Mobile robot localization Biological approach to vision Gist model Testing and results Discussion and conclusion
Mobile Robot Localization Indoors: strong assumptions of flat walls, narrow hallways, and solid angles
• Ranging sensors (laser and sonar) for mapping Outdoors: less conforming set of surfaces
• Ranging sensors are less effective, vision is better
Robot Vision Localization Scene-based Vision Localization
Scenes as a whole as Landmarks Color histograms [Ulrich
and Nourbakhsh 2000] Fourier Transform
[Oliva & Torralba 2001] Wavelet pyramids
[Torralba 2003] Histogram of Dominant
features [Renniger & Malik 2004]
Gist Definition and background
Essence, holistic characteristics of an image Context information obtained within a eye saccade
(app. 150 ms.) Evidence of place recognizing cells at
Parahippocampal Place Area (PPA) Biologically plausible models of Gist are yet to be
proposed Nature of tasks done with gist
Scene categorization/context recognition Region priming/layout recognition Resolution/scale selection
Human Vision Architecture Visual Cortex:
Low level filters, center-surround, and normalization
Saliency Model: Attend to pertinent
regions Gist Model:
Compute image general characteristics
High Level Vision: Object recognition Layout recognition Scene understanding
Gist Model Utilize the same Visual Cortex raw features
in the saliency model [Itti 2001] Gist is theoretically non-redundant with Saliency
Gist vs. Saliency Instead of looking at most conspicuous locations
in image, looks at scene as a whole Detection of regularities, not irregularities Cooperation (Accumulation) vs. competition
(WTA) among locations More spatial emphasis in saliency Local vs. global/regional interaction
Gist Model Implementation
V1 Raw image feature-Maps Orientation Channel
• Gabor filters at 4 angles (0,45,90,135) on 4 scales = 16 sub-channels
Color:• red-green and blue-yellow
center surround each with 6 scale combinations = 12 sub-channels
Intensity• dark-bright center-surround
with 6 scale combinations = 6 sub-channels
= Total of 34 sub-channels
Gist Model Implementation
Dimension Reduction Original:
34 sub-channels x 16 features = 544 features
PCA/ICA reduction: 80 features
• Kept >95% of variance
Gist Model Implementation
Dimension Reduction Original:
34 sub-channels x 16 features = 544 features
PCA/ICA reduction: 80 features
• Kept >95% of variance
Place Classification Three-layer neural
networks
Testing & Results Site selection:
Different challenges appearance-wise Variability in area covered/ path lengths
Various lighting conditions Single-view filming Clean break between segments Scalability: combine all sites
Discussion & Conclusion Result of current model:
Success rate between 82.48% and 87.93% Combined rate of 85.96% 4.73% error in inter-site classification
Integrating saliency for robot navigation Localization within segment
• Identifying discriminating cues in the environment• Issues in object-based systems still applies
Bad view detection• Foreground objects sometimes occlude whole view
Obstacle avoidance, exploration, etc.
Discussion Integration of gist and saliency in general
Single representation of both models Influence of saliency to gist and vice versa
• Involvement of saliency in improving gist estimation• Gist helpful in identifying/filtering salient location
Testing the limits of Gist: psychophysics experiments
• Change blindness test for large scale layout changes• Varying exposure time• Isolation of bottom up - top down influences