APPLICATIONS OF LIGHT

FIELDS IN COMPUTER VISIONWEEK 1

REU STUDENT: AMARI LEWIS

P.H.D STUDENT: AIDEAN SHARGHI

APPLICATIONS OF LIGHT FIELDS IN COMPUTER VISION

•THE GOALS OF THIS PROJECT IS TO EXAMINE :a) OBJECT RECOGNITION

b) OCCLUSION DETECTION

c) 3D RECONSTRUCTION OF A SCENE

WHAT IS OBJECT RECOGNITION?• GIVEN TEST IMAGE, WE WANT TO DETERMINE WHAT THE IMAGE IS.

• STEPS FOR CONVENTIONAL OBJECT RECOGNITION:

• COLLECT DATASET

• DIVIDE INTO POSITIVE/ NEGATIVE DATA

• EXTRACT FEATURES (EX: SIFT..)

• TRAIN A CLASSIFIER FOR EXTRACTED FEATURES

• GIVEN A TEST IMAGE: EXTRACT THE FEATURES

• USE THE CLASSIFIER TO DETERMINE WHICH OBJECT IT IS

THE PROBLEM

• CONVENTIONAL METHODS USE 2D INFORMATION OF THE OBJECTS FOR DETECTION.

• EXAMPLE: A BOTTLE AND A PICTURE OF A BOTTLE ON A PIECE OF PAPER.

LYTRO LIGHT FIELD CAMERA

• USING THE LYTRO CAMERA WE ARE ABLE TO INCLUDE 3D INFORMATION WITH ONE SINGLE SHOT.

• THE CAMERA MANAGES TO CAPTURE ALL THE LIGHT FIELD DIRECTION, INTENSITY AND COLOR.

LIGHT FIELD CAMERAHOW IT WORKS…

STEREO

\.

• The conventional way of capturing 3d information is using stereo • Finding the correspondents

is difficult• However finding the

correspondents can be avoided…

EPI

• EPIPOLAR PLANE IMAGES

• CAN BE USED TO REPRESENT 3D INFORMATION OF THE OBJECT

• THIS PROJECT/RESEARCH IS BASED ON USING EPI TO DESCRIBE THE OBJECTS AND INCREASE THE OBJECT RECOGNITION ACCURACY

STEPS: i. CHOOSE 5 CATEGORIES FOR CLASSIFICATION

ii. COLLECT DATASET USING THE LIGHT FIELD CAMERA

iii.EXTRACT THE PHOTOS, USING THE LYTRO SOFTWARE

i. 5 CATEGORIES FOR DATASET USING LIGHT FIELD CAMERA

1.BICYCLES

2.BUS

3.TREES/FLOWERS

4.BUILDINGS

5.TRUCKS

II. COLLECT DATASET USING THE LIGHT FIELD CAMERA

Tree

III. REFOCUS THE IMAGE TO THE SPECIFIC OBJECTbicycles

ASSIGNMENT • THERE HAS BEEN A SIMILAR STUDY TO IMPROVE OBJECT RECOGNITION AND

TRACKING

• SCALE-INVARIANT REPRESENTATION OF LIGHT FIELD IMAGES FOR OBJECT RECOGNITION AND TRACKING

• ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE (SWITZERLAND)

REFERENCES

• NG, REN (2006). DIGITAL LIGHT FIELD PHOTOGRAPHY (DOCTORAL DISSERTATION).

STANFORD UNIVERSITY, STANFORD, CALIFORNIA.

THANK YOU !!