research in intelligent mobile robotics (and related topics) part 1: navigation and vision anna...
TRANSCRIPT
Research in Intelligent Mobile Robotics (and related topics)Part 1: Navigation and Vision
Anna Helena Reali [email protected] www.pcs.usp.br/~anna
Laboratório de Técnicas Inteligentes Escola Politécnica da Universidade de São Paulo
Carlos Henrique Costa [email protected] www.comp.ita.br/~carlos
Divisão de Ciência da Computação Instituto Tecnológico de Aeronáutica
MultiBot - Meeting #1, Lisboa 2003 - part I 2
Preface
This is a two-part talk about the research on Intelligent Mobile Robotics and related topics at: LTI – USP (Laboratório de Técnicas Inteligentes –
Universidade de São Paulo, Brazil) NCROMA-ITA (Laboratório de Navegação e Controle
de Robôs Móveis Autônomos – Instituto Tecnológico de Aeronáutica, Brazil).
These research groups are involved in the MultiBot cooperation project CAPES/GRICES with ISLab-IST.
MultiBot - Meeting #1, Lisboa 2003 - part I 3
LTI - EPUSP
Prof. Anna Reali 5 PhD Students
Alexandre Simões*, Reinaldo Bianchi, Valdinei Silva*, Valguima Odakura, Waldemar Bonventi.
3 Master Students Alexandre Cunha*, Antônio Selvatici, Luiz Carlos Maia
3 Undergraduate Students Rafael Pacheco, Márcio Seixas, Júlio Kawai
2 Final Course Projects
MultiBot - Meeting #1, Lisboa 2003 - part I 4
NCROMA – ITA
Prof. Carlos Ribeiro 1 PhD Student
Letícia Friske
5 Master Students Luís Almeida, Ricardo Maia, Juliano Pereira,
Esther Colombini*, Celeny Alves*
2 Undergraduate Students Lucas Gabrielli, Fábio Miranda
MultiBot - Meeting #1, Lisboa 2003 - part I 5
Intelligent Mobile Robotics
Part 1:Navigation• Map building• Localization
Perception• Computer Vision
Part II:Learning
Contents
MultiBot - Meeting #1, Lisboa 2003 - part I 6
Map Building
Our goal: Test map building algorithms in real robots fast enough? precise enough? ok for learning applications (e.g. path
learning)?
MultiBot - Meeting #1, Lisboa 2003 - part I 7
Map BuildingSildomar Takahashi, Carlos RibeiroRoberto Barra, Ricardo Domenecci, Anna Reali
“Efficient Learning of Variable-Resolution Cognitive Maps for Autonomous Indoor Navigation”.
Arleo, Millan e Floreano, IEEE-SMC, 1999. Advantages:
Complete algorithmic description Simple structure
Limitations: Assumes structure (orthogonal obstacles / walls) Reliance on dead-reckoning (but can be adapted to
more sophisticated localization)
MultiBot - Meeting #1, Lisboa 2003 - part I 8
The Basic Algorithm
1. Explores environment;
2. Once an obstacle is detected:
1. Determines obstacle frontiers either via: An a priori sensor model
A pre-trained neural net
2. Includes obstacle in the global map;
3. Defines new partition to explore. If there is none, END. Else finds route to new partition.
4. Executes route and explores new partition. Once an obstacle is detected, Step 2. Else, Step 1.
MultiBot - Meeting #1, Lisboa 2003 - part I 9
Detection of obstacle frontiers
Robô
Células Ocupadas
RetaCalculada
• Either a priori model or neural net model• Integration over time• Straight-line adjustment and correction (according to a priori actuator model)
0 y
x
My
Mx
MultiBot - Meeting #1, Lisboa 2003 - part I 10
Very “Scientific” Set-up
Walls Obstacles
Robot
3 x 3,5 m
MultiBot - Meeting #1, Lisboa 2003 - part I 11
Global Maps: Magellan, Neural net model
Map 1
Map 2
Map 3
MultiBot - Meeting #1, Lisboa 2003 - part I 12
Global Map: Pioneer, a priori model + straight-line model-based correction
MultiBot - Meeting #1, Lisboa 2003 - part I 13
Conclusions Tested algorithm (possibly with some
modifications) is a good compromise efficiency/precision for realistic applications: fast yet fairly accurate.
Next steps: Studies on simultaneous localization and
mapping (SLAM algorithms). Valguima Odakura (Anna Reali): SLAM based
on visual landmarks. Fabio Miranda (Carlos Ribeiro): Bayesian
landmark learning. Techniques for map building acceleration.
MultiBot - Meeting #1, Lisboa 2003 - part I 14
Markov LocalizationLuís Almeida, Carlos RibeiroJúlio Kawai, Anna Reali
Position estimation based on Bayesian update: Belief update based on sensor info Belief update based on action info
Sensor/actuator models and initial belief distribution: arbitrary.
Simple to implement.Computationally costly (Monte Carlo
implementation – particle filters – is a possible fix).
MultiBot - Meeting #1, Lisboa 2003 - part I 15
Markov Localization
X
p (x )
b )
X
p (x )
d )
X
p (x )
h )
X a)1
X c)1
X e)1 X2
X
p (x )
f)
X g )2
Probabilistic position grid
Action Model
Sensor Model Markov State
Estimator
pt
st
at
pt+1
odometerssonars
MultiBot - Meeting #1, Lisboa 2003 - part I 16
Monte Carlo Localization + GA OptimizationLuís Almeida, Carlos Ribeiro
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)
MC MC MCGA GA
GA on population of particles (fitness as combination of belief / particle cluster distribution)
GA on population of particles (fitness as combination of belief / particle cluster distribution)
• Basic idea: use GA to create a better set of particles for next MC update.
• Initial results: ok (in need of statistical validation).
MultiBot - Meeting #1, Lisboa 2003 - part I 17
Next Steps
Validation of GA approach. Better sensor and actuator models. Implementation in a real robot. Literature on Monte Carlo methods
(applications on signal detection and tracking): many variations to be tried...
MultiBot - Meeting #1, Lisboa 2003 - part I 18
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
MultiBot - Meeting #1, Lisboa 2003 - part I 19
Color Classification - I
Using threshold values: In the color representation space
Neural Network – MLP + backpropagation alg.:Alexandre Simões, Anna Reali
MultiBot - Meeting #1, Lisboa 2003 - part I 20
Derived Application:Alexandre Simões, Anna Reali
C1 C2 C3 C4 C5
Orange Classifier
Branco
. . .
Verde Claro
Verde Escuro
Amarelo
Laranja Claro
Laranja Escuro
R
G
B
Danificado
Orange Classifier - CEAGESP, SP
MultiBot - Meeting #1, Lisboa 2003 - part I 21
For number of clusters = 2 to Cmax, do: Apply FCM-GK (non-supervised fuzzy classifier)
to RGB image; Calculate the ratio c/s for each cluster set:
c = Cluster dimension/number of members s = Separation among clusters
Choose the cluster set, based on c/s. Show color classification result for the best
cluster set.
Non-supervised iterative fuzzy color classificationWaldemar Bonventi, Anna Reali
MultiBot - Meeting #1, Lisboa 2003 - part I 22
An example: soccer
The best cluster set 6 clusters
MultiBot - Meeting #1, Lisboa 2003 - part I 23
Another example: Rio de Janeiro
The best cluster set 3 clusters
MultiBot - Meeting #1, Lisboa 2003 - part I 24
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
MultiBot - Meeting #1, Lisboa 2003 - part I 25
Background Model - I
Background subtraction Thresholding the error between an estimate of
the image without moving objects – M(C) – and the current image:
Model can not adapt to environment changes!
M(C)Current Image
MultiBot - Meeting #1, Lisboa 2003 - part I 26
Background Model – IIMárcio Seixas, Anna Reali
Time-Adaptive, Per-Pixel Mixture-of-Gaussians: Time series of observations at a given pixel (its color) is
modeled by a mixture-of-gaussians. Based on the persistence and the variance of each of the
gaussians of the mixture, it is determined which gaussians may correspond to background colors.
Hypothesis: gaussian distributions with low variance and high persistence correspond to background model.
Per-pixel models are updated as new observations are obtained (according to a learning rate).
It is capable of dealing with long-term scene changes (e.g. lighting changes)!
MultiBot - Meeting #1, Lisboa 2003 - part I 27
Derived Application:platform occupancy
Fixedmodel M(C):
Original:
Original – M(C):
Adaptive Model:
Terminal Rodoviário de Santo Amaro TRENDS & Prefeitura de São PauloMárcio Seixas, Anna Reali
MultiBot - Meeting #1, Lisboa 2003 - part I 28
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
MultiBot - Meeting #1, Lisboa 2003 - part I 29
Optical Flow - idea
MultiBot - Meeting #1, Lisboa 2003 - part I 30
Robot (with camera) navigating in a stationary scenario. Calculation of the optical-flow divergent to estimate the time-
to-crash value in order to avoid collisions with obstacles. We are now investigating a robust method to directly
calculate the per-pixel time-to-crash value:
Vision-based robotic behavior:Antonio Selvatici, Anna Reali
Gray levels: near bright; far darkBlack: unknown distance
Original sequence Pixel time-to-crash Filtered values
MultiBot - Meeting #1, Lisboa 2003 - part I 31
Derived Application: monitoring of underground rail tracksLuiz Maia, Anna Reali
ALSTOM & Metrô de São Paulo
MultiBot - Meeting #1, Lisboa 2003 - part I 32
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
MultiBot - Meeting #1, Lisboa 2003 - part I 33
Distance-Map Calculation1. Calibration [Zhang, ICCV 99]
2. Matching – blob coloring+centroid+correlation
3. Triangulation Segmentation: based on color + distance-map.
Binocular Stereo VisionRafael Pacheco, Anna Reali
MultiBot - Meeting #1, Lisboa 2003 - part I 34
Derived Application:Outdoors Measurement
TRENDS & Prefeiturade São PauloRafael Pacheco, A. Reali
MultiBot - Meeting #1, Lisboa 2003 - part I 35
Conclusions In CV, we are now investigating:
Automatic learning of fuzzy color classifiers – Waldemar Bonventi, LTI;
A framework for high-level feedback to adaptive, per-pixel, mixture-of-gaussian background models – Márcio Seixas, LTI;
Mathematical formulation for direct and robustly calculate the per-pixel, time-to-crash values, considering a moving observer in a stationary scenario – Antonio Selvatici, LTI;
Distributed, real-time approach to calculate the optical flow, considering a stationary observer in a dynamic scenario – Luiz Maia, LTI.