fast census transform-based stereo algorithm using sse2 young ki baik* kyoung mu lee computer vision...

Fast Census Transform-based Stereo Algorithm using SSE2

Young Ki Baik*

Kyoung Mu LeeComputer Vision Lab.

School of Electrical Engineering and Computer Science

Seoul National University


FCV 2006

Contents

Stereo Vision

Census Transform Stereo Vision

Fast approaches

Experimental result

Conclusion and Future work


FCV 2006

Introduction

What is the stereo vision?

The stereo vision is the method to extract 3D information

using image from different view points.

Topographical survey

Obstacle detection

Object tracking

Face recognition


FCV 2006

Introduction

Trade off of algorithms

Algorithm for accurate results

Complex computation and iteration

Slow processing time

Unable to realize real-time system

Algorithm for fast processing time

Simple computation and no iteration

Fast processing time

Unable to realize accurate system


FCV 2006

Introduction

Fast stereo vision algorithmWindow size invariant method

Box filtering method “Box-filtering techniques”, M.J.McDonnell (CGIP-81’) “Real time correlation-based stereo : algorithm, implementations and applicat

ions”, Olivier Faugeras , Zhengyou Zhang , … (Tech.Rep.RR-2013, INRIA,1993)

Disparity range invariant methodRectangular subregioning method “Rectangular Subregioning and 3-D Maximum-Surface Techniques for Fast St

ereo Matching”, Changming Sun (CVPR-2001)

Parallel processing technique


FCV 2006

Introduction

ProblemReal images from grabbers can not assure of brightness consistency in corresponding region.

Intensity correlation method is not proper for real images.

Census transformCensus transform has been evaluated as the method robust to radiometric distortion.

J. Banks and P. Corke, "Quantitative evaluation of matching methods and validity measures for stereo vision," Int. J. Robotics Research, vol. 20, pp. 512-532, July 2001.

Heiko Hirschmller, "Improvements in Real-Time Correlation Based Stereo Vision", Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision, pp. 141-148, Kauai, Hawaii, December 2001.


FCV 2006


Census transform

Census transform converts relative intensity difference to

0 or 1 and deforms 1 dimensional vector as much as

window size of census transform.

210159998639

198170326747

45677810298

304033115109

393126130121

11111

11000

00X11

00011

00011

111111100000110001100011

Census transform window (CTW)


FCV 2006


Result of census transform

Census transform makes data of (image size * vector size).

Height

Height

WidthWidth

(Square size of CTW)-1


FCV 2006


Sum of Hamming distance

The Hamming distance of two transformed vectors with

correlation windows is used to find corresponding region.

Disparity range

Sum of Hamming distance

Right census

transformed vector

Left census

transformed vector

3D disparity space


FCV 2006


Complexity of algorithmCensus transform-based stereo vision (CTSV) has high complexity.

N : Searching window size

D : Disparity range

C : Census transform window size

Method Complexity

Conventional

stereo visionND

Census transform

stereo visionCND


FCV 2006

Fast Census Transform Stereo Vision


Census transform

Parallel processing

- SSE2

Hamming distance

8bit look-up table

Parallel processing

- SSE2

Correlation

Moving window technique

Parallel processing

- SSE2


FCV 2006


SSE2 (Streaming SIMD Extension 2)

128-bit SIMD packed integer & floating point arithmetic operation

Cache and memory management operation

Continuous memory structure is required

No advantages in separate data

XMM0

XMM1

XMM2

XMM3

XMM4

XMM5

XMM6

XMM72 x Double

16 x BYTE

8 x WORD

4 x DWORD

2 x QWROD


FCV 2006


Fast approaches (census transform)

Usage of parallel processing (SSE2)

16 pixels are loaded to XMM(SSE2 memory) and computed at once.

16

16

16

16


FCV 2006


Fast approaches (sum of Hamming distance)

Usage of 8bit look-up table (LUT)

Parallel processing : SSE2

Parallel processing is faster than 8 bit LUT

Shift & Bitwise AND Operation

8 times

1 1 0 1 0 0 1 0

Sum Up Number or 1

1 1 0 1 0 0 1 0

8- bit Look- up Table

4


FCV 2006


Fast approaches (correlation)

Moving window technique

Addition

Addition andSubtraction

Total sum of 2D window value

Initial value

(a) (b)

(c) (d)

y

x


FCV 2006


Fast approaches (correlation)

Combination of moving window and SIMD

Moving window technique for x, y-axis

SSE2 for d-axis

y

dx

SIMD

Moving Window


FCV 2006

Experimental result

EnvironmentSystem : Pentium-IV 2.4GHz

Cache memory : 512Kbyte

Camera : Stereo Mega-D (Videre design)

ConditionImage size : 320 x 240 gray stereo images

Census transform window size :5x5, 7x7, 9x9

Disparity searching range : 32

Correlation window size : 11x11


FCV 2006

Experimental result

Detail processing time

32 disparity searching range

Method

Census

Transfor

m

Window

Time (msec)

Frame

RateCT Cal HD Corr Total

Census

with

SSE2

5 5.9 17.5 5.8 29.2 34.2

7 11.6 29.6 5.6 46.8 21.4

9 35.4 52.6 4.3 92.3 10.8

Census

without

SSE2

5 34.3 24.1 14.2 72.5 13.8

7 66.2 35.1 14.1 115.4 8.7

9 208.3 66.5 14.2 289.0 3.5


FCV 2006

Experimental result

Performance of census transform stereo vision

57

9

No SSE2

SSE20

510

15

20

25

30

35

Frame Rate

Census Window Size

No SSE2SSE2


FCV 2006

Conclusion and Future work

ConclusionMoving window technique reduces processing time to constant except in transforming stage

SSE2 instructions reduces running time by 2.5 to 3 times

Possibility for faster resultSpecialized Instruction

16 bit look-up table

Fixed window size of census transform

Future workApplying real-time approach to another stereo algorithm

Combine stereo system to other applications

fast census transform-based stereo algorithm using sse2 young ki baik* kyoung mu lee computer vision...

Documents