January 14, 2014 Sam Siewert

ECEN 5043

Computer and Machine Vision

Lecture 1 – Introduction

Part-2

Biological Vision vs. Machine Vision (Why A Honey Bee is Better than HPC for CV)

Humans - 100 million

Photoreceptors – 10 billion Neurons (Cerebral Cortex)

– Brain with 100 billion Neurons

– Millisecond Transfer

– Massively Parallel Analog + Digital Computation

Synapse Match is a Challenge – 7000 Connections from 10 Billion Neurons

– 3 Year Olds Have 1015 Synapses

CPU to Digital Camera/HDD – Connects 10’s of millions of pixels

– to Several Billion transistors

– Through Sequential Logic and I/O Bus

Sam Siewert 2

960K Neurons in flight:

Learns locations,

complex odors,

colors, and shapes;

with high efficiency

(500 Watt/Kg), 0.218g

Brain plasticity for

learning,

connectedness,

concurrency,

integrated sensing,

power efficiency,

and resiliency

2012 – 8 billion?

NVIDIA GK110

28nm, (7.1 billion)

Intel MICA 22nm

(5 billion)

http://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons

http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_Law_-_2011.svg

How We’ll Do It Assessment of Theoretical Learning

– Two Mid-term Exams (1/2 way, 7/8 way)

– FINAL

Practice – 5 Labs

Application – 1 Extended Lab with your Own Design

Sam Siewert 3

Linux Lab and Desktop Options

Native Linux Installation – Ubuntu

Logitech C200 or C270 Camera(s)

OpenCV

ffmpeg

GIMP

Transformer.uaa.alaska.edu – available to all remotely

and in A219

Virtual-Box Ubuntu Installation

Beagle xM Ubuntu, Intel Terasic Atom Yocto Linux

Sam Siewert 4

Sam Siewert 5

Administrivia Lectures – PowerPoint with Camtasia – Recorded on Wednesdays in ENGR 227C, Distributed via Blackboard by Thurs Morning Introductions – Instructor (Office Hours) -

http://www.cse.uaa.alaska.edu/~ssiewert/Schedule-Spring-2014.pdf – Students (Introductions) – Let’s all join Google+ Circle (I will create and

invite you)

UAA Blackboard – http://www.uaa.alaska.edu/classes/

Personal Lab – You MUST Have Native Linux and I recommend VB-Linux Too – Either using your own Laptop – Or Using A219 Lab at UAA

UAA Beagle xM Linux Lab – A219, http://www.cse.uaa.alaska.edu/~ssiewert/cpal.html

January 14, 2014 Sam Siewert

Linux Digital Video and CV

Processing Skills

Introduction Session

Basic Lab Observations

CV is Compute Intensive

– Lower Resolution and Frame Rates (e.g. 640x480 or 320x240 at

30Hz)

– High-End is Really Intense (HPC) – E.g. 1000 Hz 4K Cameras

like http://www.idtvision.com/, or http://www.photron.com/

– Humans Seem to Saturate at 60Hz (Current Theory)

– 60Hz Stereo in HD is still a Massive Data Rate (1920x1080 x 3

bytes x 60 x 2), or about 720 MB/sec!!

We will work at Low Resolution and 30Hz, but with Both

2D and 3D

Both Binocular 3D, and RGB-Depth

Sam Siewert 7

Tutorial CV Papers – IBM DeveloperWorks

Build a compute node or small cluster application

and scale with HPC -

http://www.ibm.com/developerworks/cloud/library/cl-

cloudscaling1-hpcondemand/index.html

Explore video analytics in the cloud -

http://www.ibm.com/developerworks/cloud/library/cl-

cloudscaling3-videoanalytics/

Machine data analytics -

http://www.ibm.com/developerworks/library/bd-

mdasecurity/index.html

Sam Siewert 8

Labs

I will POST to BB and External Website on Thursdays

Read, Review, Start and Question that Weekend

Bring Questions to Office Hours Mon, Tues, Wed the Following Week

Lab Due one Week Later

This Works Great if YOU Keep Up

I will POST Lab #1 on 1/15/2014, Due on 1/26 for Full Credit, Accepted Late Until 1/30 (10% Penalty)

Sam Siewert 9

January 14, 2014 Sam Siewert

OpenCV Demos

Overview Session – Passive Computer

Vision Methods

2D & 3D Passive Computer Vision

Sam Siewert 11

Analog

Camera #1

LEFT (NIR, Visible) Linux with

OpenCV (x86, TI OMAP, Atom)

USB 2.0, PCIe

Host Channels Storage

Analog

Camera #2

RIGHT (NIR, Visible)

Linear Hough Transform

2D Skeletal Transform

3D Disparity & Depth Map Canny Edge Finding

Face Detection/Recognition

January 14, 2014 Sam Siewert

OpenNI

Overview Session – Active Computer

Vision Methods

3D Active Computational Photometry

Sam Siewert 13

Analog

Camera #1

RGB (Visible)

Altera FPGA

CVPU (Computer Vision

Processing Unit)

Mobile

Sensor Network

Processor (TI OMAP, Atom)

Networked

Video

Analytics

HD Digital

Camera Port

(Snapshot)

USB 2.0, PCIe

Host Channels

Flash

SD Card

Analog

Camera #2 (Near

Infrared)

TI DLP Light-crafter Kit http://www.ti.com/tool/dlplightcrafter

Depth Map

IR Pattern Projection

https://www.cs.purdue.edu/homes/aliaga/cs635-10/lec-structured-light.pdf

Photo credits and reference:

Dr. Daniel Aliaga, Purdue University

https://www.cs.purdue.edu/homes/aliaga/

3D Computer Vision Transforms Long Range ( > 5 meters) Using Passive Binocular Methods – Impractical to Project from a UAV or Long Range Observer

– Requires Image Registration

– Accurate Camera Intrinsic (Camera Characteristics) & Extrinsic (e.g. Baseline)

Short Range ( < 5 meters), Structured IR Light Projection for RGB-D – Compare to ASUS Xtion and PrimeSense – Off-the-Shelf

– Robust Depth Maps with Less Noise

– Showing Significant Promise to Improve CV Scene Segmentation and Object Recognition Compared to 2D

– “Change Their Perception”, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige, IEEE RAS, December 2013.

Sam Siewert 14

Noise in Passive Depth Maps

Robust Active Depth Map

“Change Their Perception”, By Xiaofeng Ren,

Dieter Fox, and Kurt Konolige, IEEE RAS,

December 2013.

Off-The-Shelf RGB-Depth Mappers

Intel Creative Camera – Windows Perceptual SDK

ASUS Xtion Short and Long Range – OpenNI

PrimeSense (Kinect Old and New) – MS SDK, ROS

Sam Siewert 15

Discrete Convolution

Sam Siewert 16

a[1,-1] a[0,-1] a[-1,-1]

a[1,0] a[0,0] a[-1,0]

a[1,1] a[0,1] a[-1,1]

Summary

Numerous MV and CV Applications – Inspection and Process Automation – MV Domain

– Interactive Systems and Augmented Reality – CV Domain

– Robotics – MV and CV

– Study of Human Vision and Vision Prosthetics – CV

2D Image Processing (Machine Vision) – Capture, Enhancement, Segmentation, Recognition

Passive 3D Computer Vision – Stereo Capture, Calibration, Enhancement, Registration, Depth

Mapping, Segmentation, Recognition

Active 3D Machine Vision (It’s Cheating!) – Structured Light Illumination and IR/Visible Capture, IR Analysis and

Depth Mapping, Visible Image Registration

– Works Between 0 and 5 Meters Well

Sam Siewert 17