Computational

Vision

Jitendra Malik, UC Berkeley

Computer Vision GroupUniversity of California Berkeley

From Pixels to Perception

TigerGrass

Water

Sand

outdoorwildlife

Tiger

tail

eye

legs

head

back

shadow

mouse

Computer Vision GroupUniversity of California Berkeley

Object Category Recognition

Computer Vision GroupUniversity of California Berkeley

Detection can be very fast

• On a task of judging animal vs no animal, humans can make mostly correct saccades in 150 ms (Kirchner & Thorpe, 2006)

– Comparable to synaptic delay in the retina, LGN, V1, V2, V4, IT pathway.

– Doesn’t rule out feed back but shows feed forward only is very powerful

EZ-Gimpy Results (Mori & Malik 03)

• 171 of 192 images correctly identified: 92 %

horse

smile

canvas

spade

join

here

Computer Vision GroupUniversity of California Berkeley

Caltech-101 [Fei-Fei et al. 04]

• 102 classes, 31-300 images/class

Computer Vision GroupUniversity of California Berkeley

Caltech 101 classification results

(By combining cues, one can get above 80% !)

Looking at People

• 3-pixel man• Blob tracking

• 300-pixel man• Limb shape

Far field Near field

Medium-field

The 30-Pixel Man

Examples of Actions• Movement and posture change

– run, walk, crawl, jump, hop, swim, skate, sit, stand, kneel, lie, dance (various), …

• Object manipulation– pick, carry, hold, lift, throw, catch, push, pull, write, type, touch, hit,

press, stroke, shake, stir, turn, eat, drink, cut, stab, kick, point, drive, bike, insert, extract, juggle, play musical instrument (various)…

• Conversational gesture– point, …

• Sign Language

Classifying Ballet Actions

16 Actions. Men used to classify women and vice versa.

What makes computer vision interesting?

• Great scientific problem– 30-50% of the brain is devoted to it– Visual perception has been richly studied– Long history with contributions from greats such as Euclid, Maxwell,

Helmholtz, Mach, Schrodinger etc

• Great engineering problem– Search on the web for images/video– Enhancing visual experiences– Essential for robotics and AI

• Finally, we are making great progress– Availability of computing resources– Large collections make possible the use of machine learning techniques– Adoption of interdisciplinary approach