image processing ip cores © evs srl, may 30, 2005

Image Processing IP cores

© eVS Srl, May 30, 2005

Confidential © 2005 eVS Srl. 2

Cooperation

Vision, Image Processing and Sound Lab

University of Verona

Department of Computer Science

Ultimodule


Research Areas

Image Processing • Image Restoration

• Feature Extraction (Edge, Corner, Region, Gradient)

Computer Vision • Mosaics and Video summarization

• Model-Based Tracking

• Stereopsis and 3D reconstruction

Pattern Recognition • Statistical techniques

• Clustering and classifications of sequences

• Object Recognition


Applications

Security and Monitoring• Video surveillance

• Intrusion detection

Industrial • Active video surveillance

• Robotic arm motion

• Measurement of points and distances

Marine• Auto docking

• Support to navigation (stabilization)

• Hazard detection

• Active video surveillance

Automotive• Active security

• Autonomous guided vehicles


Sensing The Real World

Stereo sensor to reconstruct the 3rd dimension

• Robustness to brightness changes on scene

• Real time performance

Output Image

• robust RGBZ• resolution up to 512x512• up to 100 fps


Image Processing Engine

• The image processing engine implements most of the algorithms required when an application is asked to rely on vision to make decisions and control

• Real time performance

• High computational power

• Self-contained modules

• Compact configuration

• Common interface

• High modularity for differentcontexts

• Optical (2D) and range (3D)image support

ImageProcessing Engine

RGBZ RGBZ*

+ programmable linear and non linear filters+ warping+ edges, corners, segments and curvesdetection+ segmentation+ tracking of points through a sequence ofimages


Stereopsis 1

The computational stereopsis is the process used to obtain the depth information from a pair of images coming from two cameras that see the same scene from two different points of view.

We can distinguish two sub-problems:

Corresponding points detection consists in finding which points in the left and right images are projection of the same point of the scene. The images must be rectified in order to ensure that corresponding points belong to the same horizontal line. Rectification reduces the algorithm complexity.

3D reconstruction: once the correspondences, the relative position of cameras, and the internal sensor parameters are known, the 3D position in the scene can be calculated for each point of the two 2D images.


LeftLeftviewview

DisparityDisparitybrigh = closebrigh = close

Stereopsis 2

RightRightviewview

DisparitDisparityy

hot = closehot = close


RGBZ sensor

Left image rectification

Right image rectification

Stereomatching

Zreconstruction

RGBZimage

Stereo Post processing


Project Overview

Filtering

Segmentation

Corner

Edge

Gradient

Warping

Motionestimation

Image Processing Engine:a possible configuration

RGBZSensor Post-

processing


Filtering

Noisy image (Gaussian noise σ = 0.01) Filtered image (5x5 Gaussian filter σ = 1)

Filtering

• The noise is reduced by convolving the intensity image with a gaussian kernel (linear filtering)

• The amount of filtering can be controlled by changing the coefficients of the convolution mask


Segmentation

Histogram

threshold

Segmented imageInput image

• Segmentation is usually one of the first steps in image analysis

• The purpose of image segmentation is to subdivide an image into meaningful, non-overlapping regions

• Our implementation produces as a result a binary image which distinguishes between background and foreground


Edge detection

EdgeDetection

Original image Edge

• An edge is a set of connected pixels that lie on the boundary between two regions

• To detect an edge we apply a threshold to the magnitude of image gradient

• Gradient is computed by convolving the image with a 3x3 Prewitt mask, and edges are thinned applying non-maximum suppression


Corner detection

• Corner points are detected by a significant change of the gradient values along two directions

• The core receives in input the image gradients and the required number of corners, and outputs the coordinates and the degree of confidence ofthe extracted corners.

• Corners are easy to track

• Once the position of cornersis known along the video sequence, many informations on camera motion can be retrived.


• Warping consists in applying a geometrical spatial transformation T:R2→R2 to the image coordinates, and re-sampling the grid

• Some features of our image warping module:

• affine transformation

• backward mapping

• bilinear interpolation

Warping


Summary 1

SPACE Slices BRAM DSP48s

Filtering 184 2 3

Edge-detection 1195 5 0

Corner-detection 1799 14 8

Warping 422 0 4

Segmentation 858 2 4

Rectification 1312 3 0

Stereo n.a. n.a. n.a.

Performances estimation on Xilinx Virtex4 LX60 with 8 bit depth 512x512 input images


Summary 2

TIME Max Freq (MHz) Pixel Rate (MHz) Frame rate (fps)

Filtering 197 49 200

Edge-detection 53 9 36

Corner-detection 42 21 86

Warping 10 26 105

Segmentation 91 91 370

Rectification 209 52 212

Stereo n.a. n.a. n.a.

Performances estimation on Xilinx Virtex4 LX60 with 8 bit depth 512x512 input images


Work in progress

• Motion estimation: frame to frame geometric image transformation computation

• Video stabilization: unwanted motion compensation

• 2D Feature Tracker: correlation-based tracker of a region of interest

• Hough Transform: straight lines detection


Development tools

• Matlab:• Algorithm design• Prototyping• High level verification

• Xilinx System Generator for DSP 7.1:• Hardware design• Simulation• Hardware co-simulation (via JTAG)• Initial resources estimation

• Xilinx ISE 7.1:• Implementation• Integration

• Mentor ModelSim SE 6.0:• Simulation


Development boards

• 3 Xilinx Spartan3 Starter Kit Boards• 200,000 gates Xilinx Spartan3 XC3S200FT256 FPGA

• 2Mbit Xilinx XCF02S Platform Flash

• 1M-byte of Fast Asynchronous SRAM

• 1 ML310• Virtex-II Pro XC2VP30

• 256MB DDR memory

• 512MB CompactFlash card

• multiple PCI slots

• 1 SCM40• NEC VR4131 MIPS

• DDR SDRAM

• 200,000 gates Spartan2


Contacts

eVS is:

• Prof. Vittorio Murino: [email protected]• Dott. Roberto Marzotto: [email protected]• Dott. Alessandro Negrente: [email protected]• Dott. Marco Monguzzi: [email protected]

• Phone / fax: +39 045 802 7027• Web: www.evsys.net

image processing ip cores © evs srl, may 30, 2005

Documents