Embedded Real-time Stixel Computation

Daniel Hernández Juárez, Antonio Espinosa, Juan Carlos Moure

David Vázquez, and Antonio M. López

Computer Architecture & Operating Systems Department (CAOS)Autonomous University of Barcelona

Stixel World: Compact representation of the world

Stereo of 1280 x 960 = 1,228,800 pixels => Too much data to process

Medium-level representation with only relevant information

Fixed width stixels, variable number of stixels per column

Stixel = Stick + Pixel

Stereo + Horizon Line + Road Slope Stixels

Obj.

Sky

Obj.

Obj.

Grnd

slopeHorizon

Stereo Images

Stixel World: Idea & Computational Complexity

3

Image Resolution

640 x 480 147 M ops.

1280 x 480 294 M ops.

1280 x 960 1179 M ops.

Total Computation Work

( Width × Height 2 )

Sky: Very far, all pixels = disparity near 0

Object: All pixels = Constant disparity

Ground: Disparity close to expected model (given ground estimation)

Computed independently for each column Enforces constraints: no sky below horizon, no neighbors objects at same distance…

Stixel = {vBase, vTop}, all possible vBase and vTop combinations tested => complexity is O(wh2)Combinatorial explosion (of possible configurations): Dynamic programming to evaluate efficiently

GPU Implementation: Proposal4

Stereo Disparity

First level parallelism (big granularity): Second level parallelism (fine-grained):

Independent / Task level: Typical CPU parallelizationEach image column is processed in parallel by a CTACTA = Cooperating Threads Array

Block of threads computes collaboratively

CTA

···

…

CTA

h

h

GPU Implementation: Second Level Parallelism5

CTA

step 1

CiCi

CiCi

step 2 step 3 …..

• Extra parallelism level needed for efficient GPU use• Sequentially perform h (image height) steps• CTA threads collaborate sharing info each step• Decreasing Parallelism: Each step uses one thread fewer

Second level parallelism (fine-grained):

h

Computational AnalysisThread Parallelism h×w

Compute Work per thread h

Total Global Data Reads h2×w

Total Global Data Stores h×w

More details

step h

…..

GPU Implementation: Performance & Energy Efficiency6

GPU Performance (Frames/Second, fps) Energy Efficiency

• Real-time performance for energy efficient GPU: NVIDIA Drive PX

• NVIDIA Drive PX has better energetic efficiency than high-end GPUs

8,68

4,004,57

2,322,231,49

0

1

2

3

4

5

6

7

8

9

10

Tegra X1 Titan X

fram

es

per

se

con

d /

Wat

t

1280 x 240

640 x 480

1280 x 480

86,8

1000

45,7

581,0

22,3

373,0

1,0

10,0

100,0

1000,0

Tegra X1 Titan X

fram

es

per

se

con

d

1280 x 240

640 x 480

1280 x 480

Thank you

Daniel Hernández Juárez, Antonio Espinosa,

Juan Carlos Moure, David Vázquez,

and Antonio M. López

Autonomous University of Barcelona

More information:www.cvc.uab.es/people/dhernandez