computer vision for robotic navigation and augmented reality applications

8/9/2019 Computer vision for robotic navigation and augmented reality applications

1/44

Introduction

Computer vision by phase encoding methods

Developed phase-based measurement system

Conclusions

References

Computer vision for robotic navigation and augmentedreality applications

Dr. Rigoberto Juarez Salazar

1Instituto Tecnológico Superior de ZacapoaxtlaDivisión de Ingenierı́a Informática

Congreso Multidisciplinario Ciencia y Tecnologı́a para elDesarrollo Sustentable

October 15, 2014

1 / 4 0 Dr. Rigoberto Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications

http://find/


2/44

Introduction



Conclusions

References

Content

1 Introduction

2 Computer vision by phase encoding methods

3 Developed phase-based measurement systemNormalización de patrones de franjasFourier fringe-normalized analysisInhomogeneous phase-shiftingDesenvolvimiento de fase

4 Conclusions

5 References


http://find/http://goback/


3/44

Introduction



Conclusions

References

Introduction

Figura : Metrology is a very important task in most of human task. It increases our senses,particularly, vision sense. Vision is about discovering from images what is present in the scene and

where it is. It is our most powerful sense.


http://find/


4/44

Introduction



Conclusions

References

What is computer vision?

In computer vision a camera (or several cameras) is linked to a computer. Thecomputer automatically interprets images of a real scene to obtain useful information(e.g., 3D reconstruction) and then acts on that information (e.g. for navigation or

manipulation).


http://find/


5/44

Introduction



Conclusions

References

What is computer vision?

In computer vision a camera (or several cameras) is linked to a computer. Thecomputer automatically interprets images of a real scene to obtain useful information(e.g., 3D reconstruction) and then acts on that information (e.g. for navigation or

manipulation).

It is not:

Image processing. Image enhancement, image restoration, image compression. Takean image and process it to produce a new image which is, in someway, more desirable.

Pattern recognition. Classifies patterns into one of a finite set of prototypes.


http://find/


6/44

Introduction



Conclusions

References

Applications of computer vision

Automation of industrial processes: object recognition, visual inspection, robothand-eye coordination, robot navigation.

Space and military: remote sensing, surveillance, target detection and tracking(traffic, aircraft, etc.), UAV localization.

Human-computer interaction: Face detection and recognition. Mobile-phoneapplications, augmented reality.

3D modeling.


I t d ti

http://find/


7/44

Introduction



Conclusions

References

A proposed computer vision-based robotic navigation


Introduction

http://find/


8/44

Introduction



Conclusions

References

Computer vision-based robotic navigation


Introduction

http://find/


9/44

Introduction



Conclusions

References

Approaches in computer vision

There are two main approaches to obtain three-dimensional information of a scene.

Amplitude or intensity modulation

The information is obtained from variations of intensity of the object, for example,

shadows, illumination conditions, surface color, etc. It is simple to implement and easythe experimental setup. However, it is very sensitive to noise sources.

Phase modulation

The information of interest is encoded into a phase distribution, for example, by phaseshifting, or frequency carrier. The experimental setup is lightly more complex than

amplitude modulation approach. The data processing is more complex. The mainadvantages is that phase modulation have a very high robustness to noise sources. Itallows to reach a high resolution.


Introduction



10/44

Introduction



Conclusions

References

Phase-based measurement systems

!"#$%&'(

*'+',-.-+ /-0$%01

2-3%&-

40*5.$ 65.*5.$ 7+%01-8

*'9-+0$

:-$5+'02

!"#$%&'$()*+ -$).#

/%&(0$ *(*+1-&-

/; /< /= />/?

!"#$%&'

)*+&"$,

-&,."*$/

0"&'

)"12&,,#$%

3"*))&/

)4*,&

&56"*271$

8$6&"9&/#*6&

)"12&,,#$%

04*,&'

.$:"*))#$%

01,6'

)"12&,,#$%

Figura : General scheme of a phase-based measurement system.

The experimental setup generates fringe-patterns where the encoded phase isassociated with the physical parameter of interest.

The fringe analysis block extracts the physical information from the givenfringe-patterns.


Introduction

http://find/


11/44

Introduction



Conclusions

References


By the good properties of using phase modulation, we are focused on reconstruction of3D objects by phase-shifting as well as Fourier fringe analysis to robotic navigation andaugmented reality applications.

The phase encoding approach consist on the following two procedures.

1. Wrapped phase extraction

One or more fringe-patterns are processed to obtain the encoded phase in a wrappedformat. For this, the two main methods are

Phase-shifting method.

Advantages. Full resolution is reached, algorithms are simple and very robust.Disadvantages. More than one image is required. This make it no much available

for real-time applications.Fourier fringe analysis.

Advantages Only one image is sufficient to work, it is appropriate for real-timeapplications.

Disadvantages. Reduced resolution, algorithms are complex. Filtering is difficult.

10 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications

Introduction

http://find/


12/44



Conclusions

References


By the good properties of using phase modulation, we are focused on reconstruction of3D objects by phase-shifting as well as Fourier fringe analysis to robotic navigation andaugmented reality applications.

The phase encoding approach consist on the following two procedures.

2. Phase unwrapping

The synthetic phase jumps induced by the phase extraction must be removed beforethat any useful information can be reached. For this, a phase unwrapping process isapplied in the pipeline.

.


Introduction

http://find/


13/44



Conclusions

References

Phase demodulation process

(a) (b) (c)

!"#$%&'(

*'+',-.-+ /-0$%01

2-3%&-

40*5.$ 65.*5.$ 7+%01-8

*'9-+0$

:-$5+'02

!"#$%&'$()*+ -$).#

/%&(0$ *(*+1-&-

/; /< /= />/?

!"#$%&'

)*+&"$,

-&,."*$/

0"&'

)"12&,,#$%

3"*))&/

)4*,&

&56"*271$

8$6&"9&/#*6&)"12&,,#$%

04*,&'

.$:"*))#$%

01,6'

)"12&,,#$%

Figura : General scheme of a phase-based measurement system.


IntroductionNormalización de patrones de franjas

http://find/


14/44



Conclusions

References

Normalizacion de patrones de franjas

Fourier fringe-normalized analysis

Inhomogeneous phase-shifting

Desenvolvimiento de fase

Fringe-pattern normalization. . .

An efficient and automatic processing algorithm.

I k

u2 sat

âk

LS +!

+

LS

b̂2

k 2 b̂

k u v2 u

! I k

I k


IntroductionNormalización de patrones de franjas

http://find/


15/44



Conclusions

References





Fringe-pattern normalization

Let be a set of K fringe-patterns of the form

I k (p ) = a k (p ) + b k (p ) cos Φk (p ), k = 0,K − 1. (1)

Then, the background and modulation lights can be recovered by1

a k = Aa A†a I k ,

b 2k = 2Ab A†b

(I k − a k )2.

(2)

Finally, the normalization is carried out by the function sat(·) as

Ī k = sat

I k − a k

b k

= cos Φk (p ) ∀b k (p ) = 0. (3)

I k

u

2 sat âk

LS +!

+

LS

b̂2

k 2 b̂

k u v

2 u

! I k

I k

Figura : Block diagram of the proposed fringe-pattern normalization method.

1Juarez-Salazar, R., Robledo-Sanchez, C., Meneses-Fabian, C., Guerrero-Sanchez, F., and Aguilar, L. A.Generalized phase-shifting interferometry by parameter estimation with the least squares method. Optics and

Lasers in Engineering , 51(5):626 – 632 (2013).


Introduction

C i i b h di h dNormalización de patrones de franjas

http://goforward/http://find/http://goback/


16/44



Conclusions

References





Example for the two-dimensional caseFringe-pattern normalization

Video...


Introduction

C t i i b h di th dNormalización de patrones de franjas

http://find/


17/44



Conclusions

References

p j




I !"#

! w I

""#%

! w

I

&'()*+,-.

&'()*+,-.

"/0+),)1 '()*+,-.

"/0+),)1 '()*+,-.

234

254

"# " !"#

6

6

7).38/89

:),; ;,1),

Wrapped phase extraction. . .

Advanced algorithms for the extraction of wrapped phase byusing spatial and temporal carriers.

!

c A

I I sw cos

!1

LS

LS

!cS

cos! !

I

! ! w

#$% #$$%

LS


Introduction

Computer vision by phase encoding methodsNormalización de patrones de franjas



18/44



Conclusions

References






Introduction


http://find/


19/44



Conclusions

References





The classical approach considers fringe-patterns of the form

I (p ) = a (p ) + b (p ) cos[φ(p ) + 2πf · p ]. (4)

The respective spectrum is

I (µ) = A(µ) + C (µ − f ) + C ∗(µ + f ). (5)

If a fringe-pattern normalization is previously applied, both background and modulation

lights are removed and the respective spectrum is suppressed. Thus, the filteringprocedure is less critical

I !"#

! w I ""#%

! w

I

&'()*+,-.

&'()*+,-.

"/0+),)1 '()*+,-.

"/0+),)1 '()*+,-.

234

254

"# " !"#

6

6

7).38/89

:),; ;,1),

Figura : (Top) Analysis scheme with the Fourier method. (Down) Proposed scheme.2

2Casco-Vasquez, J. F., Juarez-Salazar, R., Robledo-Sanchez, C., Rodriguez-Zurita, G., Sanchez, F. G.,Arévalo Aguilar, L. M., and Meneses-Fabian, C. Fourier normalized-fringe analysis by zero-order spectrum

suppression using a parameter estimation approach. Optical Engineering , 52(7):074109 –074109 (2013).18 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications

Introduction


http://find/


20/44



Conclusions

References




Numerical example

Casco-Vasquez, J. F., Juarez-Salazar, et. al., Fourier normalized-fringe analys is by ze ro -ord er s p ec tru m suppression using a parameter estimation approach. Optical Engineering , 52(7):074109–074109 (2013).19 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications

Introduction


F i f i li d l i

http://find/


21/44



Conclusions

References




Numerical and experimental examplesWrapped phase extraction by using the proposed Fourier fringe-normalized analysis.

3Casco-Vasquez, J. F., Juarez-Salazar, R., Robledo-Sanchez, C., Rodriguez-Zurita, G., Sanchez, F. G.,Arévalo Aguilar, L. M., and Meneses-Fabian, C. Fourier normalized-fringe analysis by zero-order spectrum

suppression using a parameter estimation approach. Optical Engineering , 52(7):074109 –074109 (2013).20 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications

Introduction


Fourier fringe normalized analysis



22/44

p y p g


Conclusions

References




Improved efficiency and robustnessFourier fringe-normalized analysis

Video...


Introduction



http://find/


23/44


Conclusions

References




Inhomogeneous generalized

phase-shifting algorithm


IntroductionComputer vision by phase encoding methods

Normalización de patrones de franjas


http://find/


24/44


Conclusions

References




Inhomogeneous generalized phase-shifting algorithm

For phase-shifting, it are considered fringe-patterns of the form

I k (p ) = a k (p ) + b k (p ) cos[φ(p ) + δk (p )], (6)

where the phase shift δk (p ) is, in general, inhomogeneous and unknown .Video...

xy

P h a s e

s h i f t δ

(a)

xy

P h a s e

s h i f t δ

(b)

xy

(c)

P h a s e

s h i f t δ

xy

(d)

P h a s e

s h i f t δ

Figura : Generalized phase shift. Homogeneous: (a) linear, and (b) nonlinear in k . Inhomogeneous:(c) nonlinear in k but linear in p , and (d) nonlinear in both k and p .



D l d h b d



http://find/


25/44


Conclusions

References




Algoritmo de corrimiento de fase generalizado inhomogéneoDescripción del algoritmo

1 Normalización de los patrones de franjas.2 De las cantidades Ak = Ī k −1 + Ī k y S k = Ī k −1 − Ī k de dos patrones adyacentes,

el coseno del paso de fase entre esos patrones se obtiene mediante

c Ak = Ac A†c (A

2k − 1), (7a)

c Sk = Ac A†c (1 − S

2k ). (7b)

3 La función Γ(·) [ver Fig. 8(b)] se usa para seleccionar entre c Ak y c Sk comocosαk = c Sk Γ(ωc k ) + c Ak Γ(−ωc k ), (8)

donde el paso de fase αk se obtiene calculando el coseno inverso de (8).

−3

−2

−1

0

1

2

3

(a)

0 π /4 π /2

Phase step αk

3π /4 π −1 −0.5 0 0.5 1

0

0.2

0.4

0.6

0.8

1

ck

(b)

−3

−2

−1

0

1

2

3

(c)

0 π /4 π /2

Phase step αk

3π /4 π

A2 − 1

1 − S2

cos(α

k ) Γ(−ω c k )

Γ(ω ck )

Equivalent data

cos(αk )

k

k

Figura : (a) Los datos A2k − 1 y 1 − S 2k para estimación de cos αk . Para αk ∈ [0, π/2] es

conveniente elegir 1 − S 2k porque la amplitud del ruido es baja, y viceversa para αk ∈ [π/2, π].24 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications


D l d h b d t t





26/44


Conclusions

References

g y



Algoritmo de corrimiento de fase generalizado inhomogéneo

4 El corrimiento de fase es recuperado mediante la sumatoria:

δk = δ0 +k

=1

α, k = 1, 2, · · · ,K − 1. (9)

5 finalmente, la fase envuelta es obtenida mediante

φw = arctan(ζ/ξ), (10)

donde ζ = sin φ y ξ = cosφ son obtenidos mediante:ξζ

= A†

φ

I 0 I 1 · · · I K −1

T , Aφ =

cos δk − sin δk

. (11)

!

c A

I I sw cos

!1

LS

LS

!cS

cos! !

I

! ! w

#$% #$$%

LS

Figura : Diagrama de bloques del algoritmo de corrimiento de fase generalizado inhomogéneo

propuesto.4

4R. Juarez-Salazar, et. al., Generalized phase-shifting algorithm for inhom og ene ou s pha se s hi ft an d

spatio-temporal fringe visibility variation. Opt. Express , 22(4):4738–4750 (2014).25 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications


Developed phase based measurement system



http://find/


27/44


Conclusions

References

g y



Extracción rápida, automática y robusta de fase envueltaAlgoritmo de corrimiento de fase generalizado inhomogéneo

Video...

•Corrimiento de fase generalizado •Visibilidad espacio-temporal •Solo dos patrones








28/44


Conclusions

References



Extracción rápida, automática y robusta de fase envueltaAlgoritmo de corrimiento de fase generalizado inhomogéneo

Video...

•Corrimiento de fase no-lineal inhomogéneo •Visibilidad espacio-temporal •Solo dos

patrones






http://find/


29/44


Conclusions

References



Desenvolvimiento de fase. . .

Desenvolvimiento de fase rápido y exacto

! !!! 1

2! round

!k LS

!k 2!

2! k

!

! +round

k






I h h hifti

http://find/


30/44


Conclusions

References




Un mapa de fase envuelto ψ(p ) puede ser descrito como

ψ(p ) = φ(p ) − 2πk (p ), k (·) ∈ Z , (12)

donde φ(p ) es el mapa de fase continuo y 2πk (p ) (con k una función de valor entero)son los saltos de fase.

Enfoque espacial de desenvolvimiento de fase

En general, los algoritmos de desenvolvimiento de fase se basan en la ecuación deItoh:

∇φ(p ) = W [∇ψ(p )]. (13)

La ecuación (13) se puede resolver para φ mediante los métodos de seguimiento de

trayectoria o de norma mı́nima:

φ(p ) = φ0(p ) +

Ω

W{ψ(p )} dp , o mı́nφ

W [∇ψ(p )] − ∇φ(p ). (14)






Inhomogeneo s phase shifting

http://find/


31/44

p p y

Conclusions

References




Usualmente, el desenvolvimiento de fase se realiza removiendo los saltos de fase yreconstruyendo el mapa de fase continuo.

φ(p ) = ψ(p ) + 2πk (p ), (15)

Alternativamente, es posible reconstruir los saltos de fase y entonces agregar lainformación de fase envuelta para obtener un mapa de fase continuo.

φ(p ) = ψ(p ) + 2πk (p ). (16)

Algunas ventajas del método alternativo

∇ψ(·) ∈ , ∇k (·) ∈ D ={−1,0, 1} Caso discreto,

{−∞, 0,∞} Caso continuo. (17)

Esto sugiere algoritmos más simples y robustos debido a que es directo y estableaproximar a un elemento de tres posibles.






Inhomogeneous phase shifting

http://find/


32/44

Conclusions

References



Desenvolvimiento de fase por mı́nimos cuadrados y redondeo

Se puede mostrar que el gradiente ∇k (p ) satisface la ecuación

∇k (p ) = round

1

2π∇ψ

, (18)

donde round(·) indica la operación de redondeo. Entonces, la función k se puedeestimar desde ∇k mediante el problema de optimización:

m ı́nk̃ k x − k̃LT x 2

F

+ k y − Ly k̃ 2F , (19)

donde k̃ es la función que aproxima a los datos k .

El problema de optimización (19) es equivalente a resolver la ecuación de Lyapunov[?] Ak̃ + k̃ B = C , con A = LT y Ly , B = L

T x Lx , y C = L

T y k y + k x Lx .

! !!!

1

2! round !k LS

!

k 2! 2! k

!

! +round k

Figura : Desenvolvimiento de fase por mı́nimos cuadrados y redondeo.5

5R. Juarez-Salazar, et. al., Phase-unwrapping algorithm by a rounding-lea st- squ ar es ap pro a ch . Op t.

Engineering , 53(2):074109 (2013).31 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications






http://find/


33/44

Conclusions

References



Desenvolvimiento de fase rápido, automático y exactoDesenvolvimiento de fase por mı́nimos cuadrados y redondeo

Video... •Funcionamiento del algoritmo de desenvolvimiento de fase propuesto




C




http://find/


34/44

Conclusions

References

Inhomogeneous phase shifting



Video... •Factibilidad del algoritmo propuesto




C l i






35/44

Conclusions

References

o oge eous p ase s t g



Figura : Ejemplos de desenvolvimiento de fase. (1ra columna) Datos sintéticos, (2da y 3racolumna) Datos experimentales obtenidos por interferencia, (4ta columna) Datos experimentales

obtenidos por proyección de franjas.




Conclusions






36/44

Conclusions

References

g p g



Reconstrucción de objetos 3D mediante proyección de franjas usando el algoritmo dedesenvolvimiento de fase propuesto.




Conclusions






37/44

Conclusions

ReferencesDesenvolvimiento de fase






Conclusions






38/44

Conclusions



Reconstrucci´on de objetos 3D mediante proyecci

´on de franjas usando el algoritmo dedesenvolvimiento de fase propuesto.




Conclusions




D l i i d f



39/44

Conclusions







Conclusions




D l i i t d f

http://find/


40/44




(a) (b) (c)




Conclusions

http://find/


41/44

References

Conclusions

General conclusions

1 A fringe analysis scheme to phase demodulation for fast and automaticapplications was presented.

2 The simplicity of the developed algorithms, makes possible the implementation ofthe fringe analysis toolbox in dedicated hardware to address real-time

applications.

3 By the flexibility of the proposed scheme, it may be implemented in many othermeasurement areas such as machine vision and adaptive optics.

Particular conclusions

1 Like the Fourier fringe analysis method, the proposed fringe-pattern and thegeneralized phase-shifting algorithms may be extended to more than two spatialdimensions.

2 For the proposed generalized phase-shifting algorithm, if the optical setup is staticand the object is dynamic, the computed phase shift will correspond to the object.It may be useful to extract the phase object without a phase shift. An application of

this may be optical coherent tomography.37 / 40 Dr. Rigober to Juárez Salazar [email protected] Computer vision in robotic navigation and augmented reality applications



Conclusions

http://find/


42/44

References

References I

R. Juarez-Salazar, C. Robledo-Sanchez, C. Meneses-Fabian, F. Guerrero-Sanchez, and L. A. Aguilar,

“Generalized phase-shifting interferometry by parameter estimation with the least squares method,” Optics and Lasers in Engineering 51, 626–632 (2013).

Robledo-Sanchez, C., Juarez-Salazar, R., Meneses-Fabian, C., Guerrero-Sánchez, F., Aguilar, L. M. A.,

Rodriguez-Zurita, G., and Ixba-Santos, V. “Phase-shifting interferometry based on the lateral displacement ofthe light source” Opt. Express , 21(14):17228–17233 (2013).

Casco-Vasquez, J. F., Juarez-Salazar, R., Robledo-Sanchez, C., Rodriguez-Zurita, G., Sanchez, F. G.,Arévalo Aguilar, L. M., and Meneses-Fabian, C. “Fourier normalized-fringe analysis by zero-order spectrumsuppression using a parameter estimation approach” Optical Engineering , 52(7):074109–074109 (2013).

Gannavarpu Rajshekhar and Pramod Rastogi “Fringe analysis: Premise and perspectives” Optics and Lasers

in Engineering , 50(8):iii–x (2012).

J. H. Bruning, D. R. Herriott, J. E. Gallagher, D. P. Rosenfeld, A. D. White, and D. J. Brangaccio, ”Digital

wavefront measuring interferometer for testing optical surfaces and lenses,.Appl. Opt. 13, 2693-2703 (1974).

K. Creath, ”Phase measurement interferometry techniques,ı̈n ”Progress in optics,”vol. 26, E. Wolf, ed.

(Elsevier Science Publishers, 1988), pp. 349-393.

L. L. Deck, ”Suppressing phase errors from vibration in phase-shifting interferometry,.Appl. Opt. 48,

3948-3960 (2009).




Conclusions



43/44

References

References II

C. J. Morgan, ”Least-squares estimation in phase-measurement interferometry,.Opt. Lett. 7, 368-370 (1982).

J. E. Greivenkamp, ”Generalized data reduction for heterodyne interferometry,.Optical Engineering 23,

350-352 (1984).

G. Lai and T. Yatagai, ”Generalized phase-shifting interferometry,”J. Opt. Soc. Am. A 8, 822-827 (1991).

L. Z. Cai, Q. Liu, and X. L. Yang, ”Generalized phase-shifting interferometry with arbitrary unknown phase

steps for difraction objects,.Opt. Lett. 29, 183-185 (2004).

X. Xu, L. Cai, H. Yuan, Q. Zhang, G. Lu, and C. Wang, ”Phase shift selection for two-step generalized

phase-shifting interferometry,.Appl. Opt. 50, H171-H176 (2011).

A. Patil and P. Rastogi, .Approaches in generalized phase shifting interferometry,.Optics and Lasers in

Engineering 43, 475-490 (2005).

C. T. Farrell and M. A. Player, ”Phase-step insensitive algorithms for phase-shifting

interferometry,”Measurement Science and Technology 5, 648 (1994).

Pramod K. Rastogi. Digital speckle pattern interferometry and related techniques. John Wiley and Sons, LTD ,2001.

D. Malacara, ed., Optical shop testing (John Wiley & Sons, Inc., 2007), 3rd ed.




Conclusions

R f

http://find/


44/44

References

Thank you very much for your attention

Any question?

http://find/

computer vision for robotic navigation and augmented reality applications

Documents