visual recognition and searchrogerioferis.com/visualrecognitionandsearch2014/classes/class1.pdf ·...
TRANSCRIPT
![Page 1: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/1.jpg)
Rogerio Feris, Jan 23, 2014 EECS 6890 – Topics in Information Processing
Spring 2014, Columbia University http://rogerioferis.com/VisualRecognitionAndSearch2014
Visual Recognition and Search
![Page 2: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/2.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Instructors:
Rogerio Feris http://rogerioferis.com
Liangliang Cao http://researcher.watson.ibm.com/researcher/view.php?person=
us-liangliang.cao
Jun Wang http://researcher.watson.ibm.com/researcher/view.php?person=
us-wangjun
![Page 3: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/3.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
1970s
Early Days of Computer Vision
![Page 4: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/4.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
First Digital Camera (1975)
0.01 Megapixels
23 seconds to record a photo to cassette
![Page 5: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/5.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Datasets with 5 or 10 images
Large-Scale Experiment: 800 photos (Takeo Kanade Thesis, 1973)
![Page 6: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/6.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Today
Visual Data is Exploding!
![Page 7: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/7.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Announcement of Pope Benedict in 2005
![Page 8: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/8.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Announcement of Pope Francis in 2013
Rapid proliferation of mobile devices equipped with cameras
![Page 9: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/9.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Billions of cell phones equipped with cameras
~500 billion consumer photos are taken each year world-wide ~500 million photos taken per year in NYC alone
Hundreds of millions of Facebook photo uploads per day
Era of Big Visual Data
![Page 10: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/10.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
![Page 11: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/11.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
How to annotate large amounts of visual data?
Crowdsourcing: Amazon Mechanical Turk
ImageNet: Millions of Annotated Images, Thousands of Categories
http://www.image-net.org/
![Page 12: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/12.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
ImageNet 2012 Challenge
Impressive results obtained by deep convolutional networks
![Page 13: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/13.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
http://horatio.cs.nyu.edu/
![Page 14: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/14.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Exciting Time for Computer Vision
+ DATA
+ Computational Processing
+ Advances in Computer Vision and Machine Learning
![Page 15: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/15.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Introduction
Plan for Today
Visual Recognition and Search: Applications & Challenges
Student Introductions
Course Overview and Requirements
Syllabus Tour
Cool Data for Projects
![Page 16: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/16.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Applications
Visual Recognition and Search
![Page 17: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/17.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Check Sebastian Thrun short TED talk: http://www.ted.com/talks/sebastian_thrun_google_s_driverless_car.html CVPR 2012 talk with more details: http://techtalks.tv/talks/self-driving-cars/56391/
Self-Driving Cars
Object Recognition and Tracking – mostly 3D point clouds (laser scanner)
Passed Driver’s license test in Nevada!
![Page 18: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/18.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Driver Assistance
Collision Warning Automatic Head Light Control Lane Departure Control etc.
http://www.mobileye.com/
Check interesting talk by Amnon Shashua: http://www.youtube.com/watch?v=cg5SNoTF3Ms
![Page 19: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/19.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Seeking the strongest rigid detector (CVPR 2013) Rodrigo Benenson, Markus Mathias, Tinne Tuytelaars, Luc Van Gool
![Page 20: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/20.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Mars Exploration Rover
Autonomous Navigation
Highly textured environment – good for stereo!
Terrain Classification
Automatic Photo capture of interesting scenes
Horizontal velocity estimation in landing – feature tracking
![Page 21: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/21.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Gaming
Kinect Camera: RGB + Depth sensor
Human pose Detection and Gesture Recognition
See J. Shotton et al, Real-Time Human Pose Recognition in Parts from a Single Depth Image, CVPR 2011 (Best paper award)
![Page 22: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/22.jpg)
Watch for Abnormal Events!
![Page 23: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/23.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
What did you notice in the video?
People walking?
Vehicles driving?
Abandoned bag???
![Page 24: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/24.jpg)
![Page 25: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/25.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
![Page 26: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/26.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Surveillance
http://www.today.com/video/today/51630165/#51630165
![Page 27: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/27.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
People Search in Surveillance Videos
![Page 28: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/28.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Video Demo: http://rogerioferis.com/demos/PeopleSearch_BlurredFaces.wmv
![Page 29: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/29.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Boston Bombing Event
Suspect #1 found in 4 images in top 8 results Suspect #2 found in 3 images in top page
1071 detected faces from 50 high-res Boston images (all from Flickr)
Ability to spot a person with e.g., a white hat in a crowded scene
![Page 30: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/30.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Cashier Fraud Detection: Faked Scans (aka Sweethearting fraud) Giving-away of merchandise without charge to a "sweetheart" customer (e.g., friend, family, fellow employee) by faked product scan Q. Fan et al, Recognition of Repetitive Sequential Human Activity, CVPR 2009
![Page 31: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/31.jpg)
Like a normal field guide…
that you can search and sort
and with visual recognition
See N. Kumar et al,
"Leafsnap: A Computer
Vision System for
Automatic Plant Species
Identification, ECCV 2012
![Page 32: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/32.jpg)
Nearly 1 million downloads
40k new users per month
100k active users
1.7 million images taken
100k new images/month
100k users with > 5 images
Users from all over the world
Botanists, educators, kids, hobbyists,
photographers, …
Slide Credit: Neeraj Kumar
![Page 33: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/33.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Google Goggles
Amazon
![Page 34: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/34.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Auto-focus by Face Detection http://www.rogerioferis.com/demos/FaceTrackerRealTime.avi
TAAZ Virtual Makeover http://www.taaz.com/
![Page 35: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/35.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
IntraFace Demo (http://www.humansensing.cs.cmu.edu/intraface/)
![Page 36: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/36.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Finding Visually Similar Objects
![Page 37: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/37.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Exploring Community Photo Collections http://phototour.cs.washington.edu/
![Page 38: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/38.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Organizing Photo Collections – IBM IMARS http://researcher.watson.ibm.com/researcher/view_project.php?id=877
![Page 39: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/39.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Applications
Many More Applications…
Medical Imaging
Biometrics
Handwritten Digit Recognition
And so on ….
![Page 40: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/40.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Challenges
Does it really work?
Many problems still remain to be solved…
![Page 41: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/41.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Challenges
Slide Credit: Kristen Grauman
![Page 42: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/42.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Break
![Page 43: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/43.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Student Introductions
Introductions
Student Info Form (see website: Homework #0) Send by Monday, Jan 27
![Page 44: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/44.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Course Goals:
Understand and analyze the state-of-the-art in visual recognition and search.
Hands-on experience with cutting-edge
methods for visual recognition and search.
Note: 7:00pm – 8:50pm (3 credits)
![Page 45: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/45.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Pre-requisites:
1) Courses/Background in Computer Vision or Image Processing is mandatory. 2) Programming skills in C/C++ or Matlab are also required. Please drop the class if you don’t satisfy these requirements
![Page 46: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/46.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Course Format:
Each class will consist of:
Lecture by instructor Student presentations and discussion
Paper Reviews Solo (no groups)
Project
Final project report: Paper (4-8 pages)
![Page 47: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/47.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Grading:
Class Participation (10%)
Paper Presentations (20%)
Paper Reviews (20%)
Term Project (50%)
![Page 48: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/48.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Resources:
Check course webpage for:
Publicly Available Source Code Datasets Related Courses
![Page 49: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/49.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Presentations:
Students will be divided in groups and will have to prepare one or two presentations in class (list of presentation topics will be provided soon).
More information about the format and length/time of the presentations will be available in the course webpage.
![Page 50: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/50.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Presentations: Presentation slides will have to be sent to instructors at least
one day before the presentation.
Grading based on clarity, organization, knowledge, and whether the presentation meets the time constraints.
Check http://www.slideshare.net/cameraculture/how-to-give-a-good-talk
![Page 51: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/51.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Paper Reviews: Individual evaluation (no groups)
Each review (1-2 pages) should address paper strengths,
weaknesses, quality of experiments, and describe ways in which the paper could be extended.
Check the course webpage for more details
![Page 52: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/52.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Projects:
Groups of two or three depending on the total number of students attending the class.
Students are expected to spend significant time on their projects
Project Proposal, Project Update I, Project Update II, Final Presentation, and Final Project Report (4-8 pages)
![Page 53: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/53.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Course Overview
Projects:
Projects will be graded according to the following criteria:
![Page 54: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/54.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Syllabus Tour
Broad set of topics; focus on the state-of-the-art
Check the schedule and required reading for each class in the course webpage
![Page 55: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/55.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Part I: From Low-level to Semantic Visual Representations
![Page 56: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/56.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Low-Level Representation: Feature Detection and Description
Modern Descriptors for Real-Time Applications: FAST, BRISK, …
Check ECCV 2012 Tutorial on Modern Features: https://sites.google.com/site/eccv12features/slides
Classical Descriptors: SIFT and SURF
![Page 57: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/57.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Mid-level feature coding and pooling
Classical Bag-of-Word Models Spatial Pyramid Matching
Modern higher-level representations:
Fisher vector and super-vector encoding, LLC, … [K. Chatfield et al, The devil is in the details: an evaluation of recent feature encoding methods, BMVC 2011]
![Page 58: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/58.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Encoding Structure: Part-based Models
Deformable Part Models (Felzenszwalb et al)
Structureless Rigid
Poselets (Bourdev et al)
Check ICCV 2013 Tutorial on Part-based models for Recognition: http://www.cs.berkeley.edu/~rbg/ICCV2013/
![Page 59: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/59.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
High-level Representation: Attributes and Semantic Features
Output of Semantic Classifiers as Features, Zero-shot Learning, …
Check CVPR 2013 Tutorial on Attributes: https://filebox.ece.vt.edu/~parikh/attributes/
![Page 60: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/60.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Part II: Tools for Large-scale Image Classification and Retrieval
![Page 61: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/61.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Large-Scale Classification
Insights on how to handle huge amounts of data and categories
Deep Convolutional Neural Networks
Check NIPS 2013 Tutorial on Deep Learning: http://cs.nyu.edu/~fergus/presentations/nips2013_final.pdf
![Page 62: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/62.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Fast Indexing and Image Retrieval
Metric Learning
Indexing via Learning-based Hashing
Database
![Page 63: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/63.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Active Learning
![Page 64: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/64.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Part III: Case Studies
![Page 65: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/65.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
Case Studies
IBM Smart Vision Suite (SVS) IBM Multimedia Analysis and Retrieval System (IMARS)
![Page 66: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/66.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Syllabus Tour
NOT Covered:
Basic Machine learning algorithms. We assume you are familiar with e.g., SVMs, …
Computer vision fundamentals. We assume you are familiar with e.g., edge detection, …
![Page 67: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/67.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Citizen Science Projects
Cool Data for Projects
![Page 68: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/68.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Cool Data for Projects
http://www.youtube.com/watch?v=AUL03ivS8bY http://www.snapshotserengeti.org/
Snapshot Serengeti
![Page 69: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/69.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Cool Data for Projects
5 Terabytes of annotated data
![Page 70: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/70.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Cool Data for Projects
Example Project: Build a web demo for fine-grained categorization of animals, using
state-of-the-art visual learning techniques
Top classification results and associated confidence: 1. Zebra (0.98) 2. Striped Hyena (0.01) 3. Leopard (0.01)
We expect a comprehensive experimental analysis for each project
![Page 71: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/71.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Past 2013 Projects
Check the previous year project presentations
![Page 72: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/72.jpg)
Visual Recognition And Search Columbia University, Spring 2014
How to Invent:
See http://www.slideshare.net/cameraculture/raskar-ideahexagonapr2010
![Page 73: Visual Recognition and Searchrogerioferis.com/VisualRecognitionAndSearch2014/classes/class1.pdf · clouds (laser scanner) Passed Driver’s license test in ... Lecture by instructor](https://reader034.vdocuments.mx/reader034/viewer/2022043018/5f3a65c8b4abb422e102acef/html5/thumbnails/73.jpg)
Visual Recognition And Search Columbia University, Spring 2014
Next Steps
Don’t forget to submit Homework #0 (Student Info Form) if you are taking this class
Background in computer vision + coding skills in
Matlab or C/C++ are strict requirements for taking the course
If you don’t satisfy these requirements, please drop the class as soon as possible, as it will help us to know the final number of students in class.
Thank you!