ecen 5043 computer and machine...

14
January 14, 2014 Sam Siewert ECEN 5043 Computer and Machine Vision Lecture 1 Introduction Part-1

Upload: others

Post on 14-Jun-2020

15 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

January 14, 2014 Sam Siewert

ECEN 5043

Computer and Machine Vision

Lecture 1 – Introduction

Part-1

Page 2: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

The Course

An introductory course on computer vision and machine vision. Topics covered include difference between computer and machine vision, image capture and processing, filtering, thresholds, edge detection, shape analysis, shape detection, pattern matching, digital image stabilization, stereo ranging, 3D models from images, real-time vision systems, recognition of targets, and applications including inspection, surveillance, search and rescue, and machine vision navigation.

http://ecee.colorado.edu/~siewerts/extra/ecen5043/index_library.html

Richard Szeliski, Computer Vision: Algorithms and Applications, Springer, 2011. (ISBN 978-1-84882-934-3) author link

Gary Bradski, Adrian Kaehler, Learning OpenCV, 2nd Edition, O’Reilly, 2012. (ISBN 978-1449314651) publisher link

Sam Siewert 2

Page 3: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Dr. Sam Siewert UC Berkeley – Philosophy/Physics

University of Notre Dame, BS - Aerospace/Mechanical Engineering

Johnson Space Center, U. of Houston – UHCL Computer Engineering, Mission Control Center

U. of Colorado, Boulder, MS/PhD – JPL, Colorado Space Grant, Computer Science

CU Boulder Senior Instructor, Adjunct Professor, CTO, Architect, Developer/Engineer in Local Start-ups

U. of Alaska, Anchorage, Assistant Professor, Computer Systems Engineering, Alaska Space Grant

Embry Riddle Prescott, Assistant Professor, CESE, Professor Adjunct, CU-Boulder

Sam Siewert 3

1984-85

1985-89

1989-92

1992-2012

2012-14

Page 4: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Sam Siewert 4

Related Industry Background

General Experience (~25 Years in Embedded and Scalable Systems) – 12 Years NASA JSC, NASA JPL / CU, Ball Aerospace

– 12+ Years Commercial Telecomm, Storage/Networks, Embedded, Digital Video

Instrumentation and Machine Vision – Spitzer Space Telescope

– Unmanned Aerial Systems

– Robotics at CU-Boulder, Arctic Sensor Systems at U. of Alaska Anchorage

Software Engineering – NASA Johnson and JPL (Shuttle Ascent/Entry Guidance, Deep

Space)

– Intel, Emulex, Start-ups

Consulting – Graphics, Storage and Networking, UAS/UAV

Page 5: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Course Topics Computer Vision – Emulation and Replication of Human-like Vision with Computers

– Goal is to Understand, Model, and Augment Human Vision

– Provide Robotics with Human-like Vision Capability

Machine Vision – Using Digital Cameras to Automate a Process (E.g. Printed Circuit

Board Inspection, Sorting Recycling, Security Cameras)

– Compared to Computer Vision (More Practical Application)

– Applications (Optical Navigation, Sorting, Segmentation and Recognition)

Digital Media – Digital Video Encoding/Decoding

– Some Overlap with A490 Course, but Only to Extent Needed to Understand CV/MV Cameras and Transport

Linux-based Labs

Sam Siewert 5

Page 6: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Computer Vision

Sam Siewert 6

http://en.wikipedia.org/wiki/File:CVoverview2.svg

Page 7: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Sam Siewert 7

Why Work with Linux and Virtual Box… From Mobiles to Super-Computing to Datacenters

From Android Mobiles to GIS and Digital Video Services

Huge Value in Open Source Drivers, Tools, and Applications – Speeds Up Time to Market

Oracle Virtual Box – Great Cross OS Test Environment

http://www.top500.org/

Tianhe – 33+ Pflops Google Embedded/Mobile

Page 8: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Why OpenCV

Long History of Computer Vision Capture in one C/C++

Library

Open Source

Runs on Linux (Easy Ubuntu install)

Students and Instructors Love it!

Abstracts Low-Level Algorithmic Details

– We will Leverage, but Also Implement Ground-Up to Learn

– Provides Algorithm Compare and Abstracted CV Design

Well Documented on the Web and in Books

Sam Siewert 8

Page 9: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Machine Vision Systems

Camera Basics – Extrinsic and Intrinsic

Embedded Systems for Machine Vision

Fundamentals – Background Elimination

– Edge Enhancement and Other Convolutions

Optical Navigation – Segmentation Methods

– Tracking (Centroid of Object)

Stereo Vision – Distance Estimation Methods (Disparity)

– Binocular Vision vs. RGB-Depth Mappers (PrimeSense, Asus Xtion, Creative Cam)

Sam Siewert 9

Page 10: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Digital Media Systems

Embedded Media Devices – Set-Top Boxes (Linux)

– Mobile Media Systems: Smart Phones, Tablet Computing, Readers, Notebooks, DVD Players, iPODs, etc.

– Digital Camera Systems (SD, HD, HD-SDI, 2K, 4K, 6K)

Resolutions/Formats - http://en.wikipedia.org/wiki/File:Vector_Video_Standards2.svg

– Game Consoles: X-box, PS3, Etc.

– Gesture Recognition, Augmented Reality

– SD , HD Cameras and Interfaces: Composite, S-Video, Component, DVI, HDMI

Scalable Digital Media Server Systems (Head End) – Post Production for Digital Cinema, TV, Web

2K, 4K, 6K Streams from Digital Cameras

Frame/Color Editing, CGI (Computer Generated Imagery), Soundtrack, Write to Distribution Media

– Digital Cinema: HD Digital Projectors, 3D Digital Projectors

– Closed Circuit Security Systems: Multi-Camera NTSC/HD

Sam Siewert 10

Page 11: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

MV vs CV vs Video Analytics Machine Vision – Photometers Used in Process Control

– Successful History

– Industrial Automation and Robotics

– Controlled Environments

– Inspection, Optical Navigation, Medical

Computer Vision – Emulate Human Vision System

– Early Underestimation – Marvin Minsky Summer Project

– Challenge of Un-controlled Environments

– 50 Years Later, Challenges Better Understood

– Vision Prosthetics, General Automation

– Recent Breakthroughs – USC, DARPA Artificial Retina, Google

Car

– Efficiency and Generalization?

Video Analytics – CV from RT or Stored/Networked Video

Sam Siewert 11

Spitzer – JPL/Caltech

CU-Boulder ECEN 5623

Page 12: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

If Possible, CV => MV Conversion – Cheat!

Practical Solution – Convert CV to MV Problem

– Loss of Generalization (Solves One Problem Rather than Class)

– Controlling Environment May Be Difficult

– Use Non-Visible Spectrum to Advantage (e.g. FLIR)

– Sensor Fusion (Visible + FLIR, RADAR, GPS, …)

– Models and Prior-Knowledge of Problem Exploited

Overhead Camera

Dark Background

Overhead Lighting

Game State / Grid Known

Shape Database

Active RGB-D (e.g. Kinect)

Sam Siewert 12

CU-Boulder ECEN 5623

Page 13: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Why is Human Vision > Computer?

Sam Siewert 13

Approximately

100+ Mega-Pixel

(Rod/Cone Count)

Cortex=10 Billion Neurons

(High fan-out)

Total=100 Billion Neurons

Neuroscience. 2nd edition.

Purves D, Augustine GJ, Fitzpatrick D, et al., editors.

Sunderland (MA): Sinauer Associates; 2001.

http://www.ncbi.nlm.nih.gov/books/NBK10848/

Red Epic 645

63 Mega-Pixel

I/O Bus (x16 5Gbps = 8GB/sec)

Camera

Link

Interface Card

Local Bus

CPU CPU

Memory

Controller

5 To 7 billion transistors 1. Neuron > Transistor

2. Better Programming? ROM?

3. More Richly Interconnected

4. Storage + Processing

> 1 Trillion Synapses

Page 14: ECEN 5043 Computer and Machine Visionecee.colorado.edu/~siewerts/extra/ecen5763/ecen5763_doc/Lecture… · If Possible, CV => MV Conversion – Cheat! Practical Solution – Convert

Biological Vision vs. Machine Vision (Why A Honey Bee is Better than HPC for CV)

Humans - 100 million

Photoreceptors – 10 billion Neurons (Cerebral Cortex)

– Brain with 100 billion Neurons

– Millisecond Transfer

– Massively Parallel Analog + Digital Computation

Synapse Match is a Challenge – 7000 Connections from 10 Billion Neurons

– 3 Year Olds Have 1015 Synapses

CPU to Digital Camera/HDD – Connects 10’s of millions of pixels

– to Several Billion transistors

– Through Sequential Logic and I/O Bus

Sam Siewert 14

960K Neurons in flight:

Learns locations,

complex odors,

colors, and shapes;

with high efficiency

(500 Watt/Kg), 0.218g

Brain plasticity for

learning,

connectedness,

concurrency,

integrated sensing,

power efficiency,

and resiliency

2012 – 8 billion?

NVIDIA GK110

28nm, (7.1 billion)

Intel MICA 22nm

(5 billion)

http://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons

http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_Law_-_2011.svg