Image Processing & Computer Vision

with MATLAB

March, 2013

Roy Fahn

Application Engineer royf@systematics.co.il Blog: www.linkedin.com/in/royfahn http://matlabisrael.blogspot.com/

Agenda

• Retinal Blood Vessel Extraction Using Kirsch's Templates

• Circular Objects Detection & Visualization • Viola–Jones Object Detection • Kanade-Lucas-Tomasi Point Tracking • Break • Image Registration using SURF • Speeding-Up Image Processing Applications using

the GPU • Video Focusing in Real-Time

The Human Eye

Retinal Scan Process

Retinal Scan Output

Applications- Retinal Diseases

• Arterial Macroaneurysms

Arterial Macroaneurysms – Risk Factors

• Often occur in patients with high blood pressure and other forms of vascular disease, such as heart attacks and strokes.

Solution: Keep your blood pressure low !

• Occur late in life

Solution: Stay young !

• Tend to occur more commonly in women than men (3:1 ratio)

Solution: Undo !

Applications - Retinal Identification

Pros: Cons:

- Accurate - Cost

- Fast - Astigmatism

- Ergonomics

Sobel Edge Detection

G > th Edge !

Sobel Edge Detection - Drawback

Chapman et al., 2001:

• Three automated techniques of edge detection to identify the boundaries and corresponding widths of retinal blood vessels

• Sobel was found to be the most inconsistent.

• Possibly relates to the program identifying the central light reflex from the blood vessel as an edge.

Chapman,N.,Witt, N.,Gao,X.,Bharath, A.A., Stanton, A.V., Thom,S.A.,Hughes,A.D.,2001. Computer algorithms for the automated measurement of retinal arteriolar diameters. Br. J. Ophthalmol. 85,74–79.

Kirsch Operator

• A non-linear operator that finds the maximum edge strength in a few predetermined directions.

• The operator is calculated as follows for directions with 45° difference:

where the direction kernels

and so on…

Kirsch,R.A.,1971. Computer Determination of the Constituent Structure of Biological Images. Computers and Biomedical Research 4,315-328.

Kirsch Implementation

• LISP

• MATLAB help

Kirsch Implementation

• MATLAB Central

Sobel Vs. Kirsch

Speeding Up MATLAB

• MATLAB Coder

• Parallel Computing Toolbox

MATLAB Coder

Parallel Computing Toolbox

Solve computationally and data-intensive problems using:

• multicore processors

• GPUs

• computer clusters (requires MDCS)

GUI Creation & MATLAB Compiler

MATLAB Compiler Vs. MATLAB Coder

Compiler Coder

VVV V Supported Functions

V C/C++ Source Code Generation

V Numeric Algorithms Speed Up

V GUI & Graphics Deployment

V MCR Needed

Wavelet Compression

Eikelboom,R., Yogesan,K., Barry,C., Constable, I., Tay-Kearney, M., Jitskaia,L., House,P., 2000. Methods and limits of digital image compression of retinal images for telemedicine. Invest. Ophthalmol. Vis. Sci. 41,1916–19 24.

Tools Used

• MATLAB

• Image Processing Toolbox

• MATLAB Coder

• Embedded Coder

• Parallel Computing Toolbox

• MATLAB Compiler

• Wavelet Toolbox

Retinal Scan - The Old Fashioned Way

Agenda

• Retinal Blood Vessel Extraction Using Kirsch's Templates

• Circular Objects Detection & Visualization • Viola–Jones Object Detection • Kanade-Lucas-Tomasi Point Tracking • Break • Image Registration using SURF • Speeding-Up Image Processing Applications using

the GPU • Video Focusing in Real-Time

Circular Objects Detection & Visualization

(Image Processing Toolbox (R2012a

והצגתםפונקציות למציאת מעגלים בתמונה נוספו

(imfindcircles ,viscircles),

והתווספה יכולת לבצע רגיסטרציה אוטומטית של תמונות בעזרת

intensity-metric optimization

(imregister ,imregconfig, (imfuse, imshowpair

Agenda

• Retinal Blood Vessel Extraction Using Kirsch's Templates

• Circular Objects Detection & Visualization • Viola–Jones Object Detection • Kanade-Lucas-Tomasi Point Tracking • Break • Image Registration using SURF • Speeding-Up Image Processing Applications using

the GPU • Video Focusing in Real-Time

System Objects

R2010a-קיימים החל מ

VIDEO.PDF-/MATLAB2010://www.systematics.co.il/hebrew/News/PDF/http

מותאמים אישית System Objectsניתן לייצר R2011b-מהחל

Cמותאמים אישית לקוד System Objectsניתן להמיר R2012a-החל מ

System Toolboxes(החל מ-R2011a)

:בהיבטים של Simulink-ו MATLABנועדו להקל על פיתוח מערכות עם

מהירות הסימולציות•

Fixed-Pointיכולת עבודה עם טיפוסי מידע מסוג •

כמות האלגוריתמים האמינים המוכנים לשימוש•

יצירה אוטומטית של קוד מתוך האלגוריתמים•

Simulink-ו MATLABבעת מעבר בין IP-שימוש חוזר ב•

System Toolboxes

Simulinkנדרש System Toolboxes-לצורך שימוש בספריות הבלוקים של ה

Computer Vision System Toolbox

:משמש לתכנון וסימולציה של מערכות ראיה ממוחשבת ועיבוד וידאו על ידי

•System Objects

VIDEO.PDF-/MATLAB2010http://www.systematics.co.il/hebrew/News/PDF/

Simulink-ו MATLAB-אלגוריתמים יעודיים חדשים לראייה ממוחשבת ב•

(Cחלקם ניתנים להמרה אוטומטית לקוד )

Simulinkבסביבת בלוקים •

Computer Vision System Toolbox (R2012a)

:בין החידושים DSP System Toolbox-אינו מצריך עוד את ה•

לצורך פעולתו Signal Processing Toolbox-וה

Integral Imageחישוב •

CAMShiftעקיבה בגישת •

MSERרים בשיטת 'גילוי פיצ•

Viola-Jonesגילוי פנים באמצעות אלגוריתם •

trainCascadeObjectDetector (R2013a)

• Requires positive and negative samples • Trains to reject negative samples as quickly as

possible • Produces an XML file compatible with OpenCV • Training can take from 5 minutes to several days

Agenda

• Retinal Blood Vessel Extraction Using Kirsch's Templates

• Circular Objects Detection & Visualization • Viola–Jones Object Detection • Kanade-Lucas-Tomasi Point Tracking • Break • Image Registration using SURF • Speeding-Up Image Processing Applications using

the GPU • Video Focusing in Real-Time

Kanade-Lucas-Tomasi (KLT)

• Algorithm for tracking points across frames (sparse optical flow)

• First introduced in 1981 (!) by Lucas and Kanade • Workflow:

– Specify initial points to track (e. g. detect corners) – Use KLT – Re-detect points if necessary

• Works well for tracking corners – detectMinEigenFeatures

– detectHarrisFeatures

– detectSURFFeatures

Agenda

• Retinal Blood Vessel Extraction Using Kirsch's Templates

• Circular Objects Detection & Visualization • Viola–Jones Object Detection • Kanade-Lucas-Tomasi Point Tracking • Break • Image Registration using SURF • Speeding-Up Image Processing Applications using

the GPU • Video Focusing in Real-Time

Computer Vision System Toolbox

:משמש לתכנון וסימולציה של מערכות ראיה ממוחשבת ועיבוד וידאו על ידי

•System Objects

VIDEO.PDF-/MATLAB2010http://www.systematics.co.il/hebrew/News/PDF/

Simulink-ו MATLAB-אלגוריתמים יעודיים חדשים לראייה ממוחשבת ב•

(Cחלקם ניתנים להמרה אוטומטית לקוד )

Simulinkבסביבת בלוקים •

MATLAB Functions in CVST (1)

Stereo Vision • Disparity* Disparity map between stereo images • epipolarLine Compute epipolar lines for stereo images • estimateFundamentalMatrix Estimate fundamental matrix from

corresponding points in stereo images • estimateUncalibratedRectification Uncalibrated stereo rectification • isEpipoleInImage Determine whether image contains epipole • lineToBorderPoints Intersection points of lines in image and image

border

Filtering • integralFilter* Integral image filter • integralImage Compute integral image • Isfilterseparable* Determine whether filter coefficients are

separable

Detection, Extraction, Matching & Tracking

• assignDetectionsToTrack Assign detections to tracks for multi-object tracking

• configureKalmanFilter Create Kalman filter for object tracking

• detectMSERFeatures Detect MSER features

• detectSURFFeatures Detect SURF features

• extractFeatures Extract interest point descriptors

• insertObjectAnnotation Insert annotation in image or video stream

• matchFeatures Find matching features

• showMatchedFeatures Display corresponding feature points

MATLAB Functions in CVST (2)

•Random Sample Consensus שערוך איטרטיבי של פרמטרי מודל מתמטי מתוך מידע נצפה הכולל חריגים–

•RANSAC לעיבוד תמונה שערוך טרנספורמציה גאומטרית–

Fundamental Matrix (Stereo vision)שערוך –

RANSAC

Fast Retina Keypoint Extractor (FREAK)

• Supported within extractFeatures

• New descriptor that outperforms SIFT, SURF

• Typically paired with corner points

• How it works

– Retinal sampling pattern motivated from Human Visual System

– Binary descriptor • Compare pair of pixel intensities

• Extract feature in 512 bits

– Use Hamming Distance in matchFeatures to match features across images

• Not yet supported with MATLAB Coder

• Reference: http://www.ivpe.com/freak.htm

a

b

c

d

G(a,b,c,d) = ‘0101’

Agenda

• Retinal Blood Vessel Extraction Using Kirsch's Templates

• Circular Objects Detection & Visualization • Viola–Jones Object Detection • Kanade-Lucas-Tomasi Point Tracking • Break • Image Registration using SURF • Speeding-Up Image Processing Applications using

the GPU • Video Focusing in Real-Time

What is a GPU?

Graphical Processing Unit

NVIDIA Quadro NVS 4200M

# Cores 48, @810 MHz

Standard Memory Config 1024 MB DDR 3

Bus Support PCI-E 2.0 x 16

History of GPUs

3D Gaming & CAD Scientific Computing

CPUs vs. GPUs

Core 1

Core 3 Core 4

Core 2

Cache

CPU (Multiple Cores)

System Memory

Device Memory

GPU (Hundreds of Cores)

Computational Fluid Dynamics

Computational Finance

Weather Modeling N-Body Simulations

Molecular Modeling

Digital Signal Processing

Problems for Running on the GPU A selection of problems from the CUDA Community Showcase:

http://www.nvidia.com/object/cuda_showcase_html.html

The Language of GPUs

Kernel Code running on the GPU

Device The card containing the GPU and memory

Host The CPU and system memory

CUDA Compute Unified Device Architecture – Language designed for general purpose use of GPU

Functionality Overview

Parallel Computing Toolbox Support for GPU Computing

Three Levels of Support:

1) Invoking built-in functions on arrays existing on the GPU

2) Translating simple MATLAB functions to run on the GPU

3) Invoking CUDA code from MATLAB

nVIDIA Solutions

Invoking Built-In Functions

GPU Device

Memory

Processing Elements

gpuArray(A)

gather(A)

Built-In Functions that Support gpuArray

Summary: Invoking Built-In Functions

32 64 128 256 512 1024 2048 409610

-4

10-3

10-2

10-1

100

101

Matrix Size (n-by-n)

Tim

e (

s)

Matrix Multiplication

CPU

GPU

32 64 128 256 512 1024 2048 409610

-4

10-3

10-2

10-1

100

101

Matrix Size (n-by-n)

Tim

e (

s)

Backslash

CPU

GPU

32 64 128 256 512 1024 2048 409610

-5

10-4

10-3

10-2

10-1

100

Matrix Size (n-by-n)

Tim

e (

s)

Array Mutliplication

CPU

GPU

32 64 128 256 512 1024 2048 409610

-5

10-4

10-3

10-2

10-1

100

Matrix Size (n-by-n)

Tim

e (

s)

Addition

CPU

GPU

Translating MATLAB Functions

GPU Device

Memory

Processing Elements

arrayfun()

bsxfun()

Supported MATLAB Code

Summary: Translating MATLAB Functions

• Accelerate scalar operations on large arrays

• Combine computations into a single kernel

CPU GPU0

0.005

0.01

0.015

0.02

CPU vs. GPU

Invoking a CUDA Kernel using feval

2rAcircle 22 4)2( rrAsquare

4

square

circle

A

A

4)P( 222 ryx

Generating Parallel Thread Execution Files

Summary: Invoking a CUDA Kernel

Pros

• Call existing CUDA code from MATLAB

• Highest level of functionality

Cons

• Requires knowledge of CUDA

• Requires knowledge of Parallel Programming

1000 1500 2000 2500 3000 3500 4000 4500 50005

10

15

20

25

30

Number of Runs

Speedup

Speedup from using a GPU Solution

Summary of Features

1) Invoking built-in functions on arrays existing on the GPU

2) Translating simple MATLAB functions to run on

the GPU

3) Invoking CUDA code from MATLAB

Ease

of

Use

G

reate

r Fun

ction

ality

2011-2012 Upgrades

• More GPU-enabled MATLAB functions & features, including capabilities from: - Communications System Toolbox (11)

- Signal Processing Toolbox (5)

- Phased Array System Toolbox

- Neural Network Toolbox

• MATLAB Compiler generated standalone executables and components now support applications that use the GPU

• Improved arrayfun function for GPU computing applications in MATLAB

• Random number generation on GPUs

Image Processing & GPU

• GPU-enabled Image Processing Toolbox functions in R2013a: imrotate, imfilter, imdilate, imerode, imopen, imclose, imtophat, imbothat, imshow, padarray and bwlookup

• User Story: Dr. Alexander Lomes - CTO, ORCASONIX Ltd.

Agenda

• Retinal Blood Vessel Extraction Using Kirsch's Templates

• Circular Objects Detection & Visualization • Viola–Jones Object Detection • Kanade-Lucas-Tomasi Point Tracking • Break • Image Registration using SURF • Speeding-Up Image Processing Applications using

the GPU • Video Focusing in Real-Time

Computer Vision System Toolbox

:משמש לתכנון וסימולציה של מערכות ראיה ממוחשבת ועיבוד וידאו על ידי

•System Objects

VIDEO.PDF-/MATLAB2010http://www.systematics.co.il/hebrew/News/PDF/

Simulink-ו MATLAB-אלגוריתמים יעודיים חדשים לראייה ממוחשבת ב•

(Cחלקם ניתנים להמרה אוטומטית לקוד )

Simulinkבסביבת בלוקים •

? Simulinkמה זה

כלי למידול וסימולציה•

(דיאגרמת בלוקים)סביבה גרפית •

Simulinkמאפיינים עיקריים של ספריה עשירה של בלוקים מוכנים•

אפשרות להוספת בלוקים ליישומים ספציפיים•

•Application Program Interface (API) פתוח

MATLABאפשרות לשילוב עם •

יכולת למדל היררכיה•

בצורה אוטומטית C/HDLאפשרות לייצור קוד •

Image Acquisition Toolbox Simulink-ו MATLABהזרמת וידאו ותמונות ישירות לתוך •

Frame grabbers-תמיכה במגוון מגוון מצלמות ו•

הפעלת התקני הצילום ושינוי המאפיינים שלהם על ידי פקודות או • (GUI)באמצעות ממשק משתמש גרפי

חיה( Preview)תצוגה מקדימה •

עיבוד פריים במקביל • להזרמת פריים חדש

Interface Windows Linux Macintosh

Analog ●

Camera Link ●

DCAM FireWire ● ● ●

GigE Vision ● ● ●

OS Integrated ● ● ●

● As of R2011b ● As of R2010b ● Pre-R2010b

הנתמכים על מערכות הממשקים (R2010b, R2011b)השונות ההפעלה

Image Acquisition Toolbox (R2012a)

•System Object בשםimaq.VideoDevice הניתן להמרה

.MATLAB Coder-באמצעות ה Cאוטומטית לקוד

(.R2013a-וב R2012b-גם ב)תוספת חומרות נתמכות •

Video Focusing in Real-Time

More Information

• Web: http://www.mathworks.com • Blog:

http://matlabisrael.blogspot.com

• LinkedIn Group: MATLAB & Simulink users in Israel

• Seminars: http://www.systematics.co.il/mathworks/Events/events.html • Courses: http://www.systematics.co.il/mathworks/Training/Courses.html

• Webinars: http://www.mathworks.com/events • Support & Sales: 03-7660111

Thank you !