ImplementingGIST

on the GPU

Refrence

• Original Work Aude Oliva, Antonio Torralba Modeling the shape of the scene: a holistic

representation of the spatial envelope International Journal of Computer Vision, Vol.

42(3): 145-175, 2001. CPU Implementation http://cvcl.mit.edu/Papers/IJCV01-Oliva-

Torralba.pdf

• Our Work Parallelize to work on the GPU

Introduction

• Recognition of real world scenes

• Spatial Envelope A very low dimensional representation of the

scene is called the Spatial Envelope. Takes care of

• Naturalness • Openness• Roughness• Expansion• Ruggedness

Intuitive Notion

• Naturalness Straight lines indicate man made structures Crooked , rough lies indicate natural

environment.

• Openness Presence of horizon reflects highly open

environment

• Roughness Refers to the size of its major components. Is correlated with the fractal dimension of the

scene

Intuitive Notion contd…

• Degree of Expansion Converging parallel lines allows to percept depth

gradient of the space. A flat view of a building would have a low DoE A street with long vanishing lines would have a high

DoE.

• Degree of Ruggedness The deviation of the ground wrt horizon Open environments have a flat horizontal ground

level Mountainous landscapes have rugged ground Ruggedness produce oblique contours, hide the

horizon. Man-made environments built on flat ground, less

rugged.

Basic Algorithm

• Create Gabor Filter Bank

• Preprocess image Local contrast normalization Local luminance invariance normalization for

color images

• Descriptor Convolve image with filters For each filter

• Divide image into blocks• Mean of each block is a number in feature vector

Parameters Involved

• Number of Scales Typically = 3

• Number of orientations per scale Typically = 8

• Image size Typically 320x240 to 1024x1024

Ways to Parallelize

• On Number of scales * orientations per scale Number of threads = 24 Less than typical image size

• On ImageMxN Pixels Number of threads = M*N >> 24. High degree of parallelism.

Parallelization for creation of Gabor filter

• Pixel Level

Threads calculating Gabor filter value

Parallelization for prefiltering image

• Pixel Level

• Fast Fourier Transforms

• Pixel by pixel Multiplications

=> Inherent Parallelism

RGB image components Filters

Descriptor

Division into blocks

Calculating Descriptor

64 128 256 512 1024

0

10000

20000

30000

40000

50000

60000

70000

CPUGPU ( Old )GPU ( New )

Graphs

Speed up

Image Size CPU Time ( ms ) GPU Time ( ms )

64 x 64 250 350 300

128 x 128 750 450 300

256 x 256 3140 800 400

512 x 512 12200 2100 550

1024 x 1024 65500 5100 1200

Thank You !!