introduction to machine vision systems
DESCRIPTION
Introduction to Machine Vision Systems. Professor Nicola Ferrier Room 3128, ECB 265-8793 [email protected]. Machine Vision. To become familiar with technologies used for machine vision as a sensor for robots. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/1.jpg)
11
Introduction toIntroduction toMachine Vision SystemsMachine Vision Systems
Professor Nicola FerrierProfessor Nicola FerrierRoom 3128, ECBRoom 3128, ECB
[email protected]@engr.wisc.edu
![Page 2: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/2.jpg)
21
Machine VisionMachine Vision
• To become familiar with technologies used for machine vision as a sensor for robots.
– Camera and lighting technology (obtaining a digital representation of an image)
– Software (computational techniques to process or modify the image data)
– Analysis/decisions: using the results of the processing in robot control
• Additional material in CS766, ECE 533, ME 739
![Page 3: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/3.jpg)
31
Machine Vision in AutomationMachine Vision in Automation
• Use a camera to inspect parts to:– Guide a robot or control automated equipment
– Support statistical analysis in a computer-assisted-manufacturing (CAM) system
– Ensure quality in manufacturing process:
• dimensions/alignment
• Determine if all components are present
• Other quality issues: color, placement, …
![Page 4: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/4.jpg)
41
Why use Vision?Why use Vision?
• Dynamic Range
• Can be remotely situated
• Passive
– emits no energy (cf. Laser, sonar, IR)
– no contact required
• Flexibility
• Affordable
![Page 5: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/5.jpg)
51
Why avoid Vision?Why avoid Vision?
• Computation
– must process images
– data = information
• Calibration
• Sensitivity to lighting conditions
/ Because the lighting is different, these 3 images appear substantially different to a computer – to a human we easily adapt our perception for variations in illumination and recognize that all three images are of the same object.
Images (arrays of pixel data) must be processed to provide information
![Page 6: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/6.jpg)
61
Example Application:Example Application:Micro-manipulationMicro-manipulation
• Micro Object handling with Micro gripper
• Postech Robotics Lab Micro gripper
Microscope Table
![Page 7: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/7.jpg)
71
A machine vision system often includes A machine vision system often includes the following elements:the following elements:
Image Acquisition (generally from a camera placed above the production line),
Image Pre-Processing (e.g. increasing the contrast, motion de-blur, etc),
Feature Extraction (e.g. measuring a distance, checking a screw is in place etc),
Decisions (i.e. is the part OK to a tolerance, is a label in the correct position), and,
Control (e.g. give the result to a Programmable Logic Controller (PLC) or robot controller).
![Page 8: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/8.jpg)
81
Image AcquisitionImage Acquisition
• Transforms the visual image of a physical objects into a set of digitized data
– Illumination
– Image formation (including focusing)
– Image detection or sensing
– Formatting camera output signal
![Page 9: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/9.jpg)
91
Image Formation and DetectionImage Formation and Detection
Image is formed by:
– Illumination flux from object
– Optics (lens)
– Photosensitive detectors (photodiodes on solid state cameras)
Vision systems have an optical-electro device that converts electromagnetic radiation from the image of the physical object into an electric signal used by the vision processing unit
![Page 10: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/10.jpg)
101
Vision – Image FormationVision – Image Formation
•Shape•Lighting•Relative Positions•Sensor sensitivity
Same shape – very different images!
![Page 11: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/11.jpg)
111
LightingLighting• Structured Lighting
• Diffuse Backlighting
• Directional backlighting
• Fiber-optic/LED ring lights
![Page 12: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/12.jpg)
121
LightingLighting
• Polarized lighting
• Oblique lighting
• Direct front lighting
• Cross polarization
![Page 13: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/13.jpg)
131
LigLightinhtin
gg• Diffuse front lighting
• Dark field illumination
• Fibre optic near in-lighting
![Page 14: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/14.jpg)
141
Image Formation and DetectionImage Formation and Detection
Light source
![Page 15: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/15.jpg)
151
Digitization of Camera SignalDigitization of Camera Signal
• Analog image data (voltage) is sampled and quantized (often to 8 bits greyscale or 24 bits of color)
![Page 16: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/16.jpg)
161
Software: Processing the Software: Processing the DataData
• The software allows the image to be processed, analyzed, and stored. – Different types of software packages are available, ranging from
easy-to-use packages with pre-defined tools, to SDKs (software development kits) that allow programmers to build custom imaging applications.
– Matlab™ has an image processing tool box
• Image Pre-processing
• Feature Extraction
![Page 17: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/17.jpg)
171
Image Pre-Image Pre-processingprocessing
• What to do with the image?
– May need to preprocess the image in order to analyze it
• Remove motion blur (ECE 533/738)
• Enhance contrast
![Page 18: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/18.jpg)
181
I Can See It – Why can’t the Computer?I Can See It – Why can’t the Computer?
• Minimize possible problems – The human eye and brain are elaborate and versatile systems, capable of identifying objects in a wide variety of conditions. For example, we are able to identify familiar people even when they are wearing different clothes, and recognize familiar landmarks when driving on a foggy day. A PC-based imaging system is not as versatile; it can only perform what it has been programmed to perform. Knowing what the system can and cannot "see" are important points to keep in mind to obtain the results you want, and reduce errors and incorrect measurements. Common variables include:
Changes in object’s color
Changes in surrounding lighting
Changes in camera focus or position
Improperly mounted camera
Environmental vibration
• A vibration-free environment with all extraneous light removed will eliminate many common problems.
![Page 19: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/19.jpg)
191
Find the man….Find the man….
Visual tasks can be made difficult!
![Page 20: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/20.jpg)
201
Distractors
Natural systems take advantage of the fact that visual tasks can be made difficult!
![Page 21: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/21.jpg)
211
I Can See It – Why can’t the Computer?I Can See It – Why can’t the Computer?
• Minimize possible problems –
– Knowing what the system can and cannot "see" are important points to keep in mind to obtain the results you want, and reduce errors and incorrect measurements.
Engineer the environment!
Great examples include commercial motion capture systems
![Page 22: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/22.jpg)
221
Feature Feature Extraction/AnalysisExtraction/Analysis
• 2D Geometric Analysis:
– Must have high contrast to separate (“segment”) part from background
• In practice back lighting is often used
• The silhouette is used to determine:
– part dimensions: Width, height, orientation, etc
– Part features (e.g. number of holes)
– Relationships between parts
![Page 23: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/23.jpg)
231
Controlled EnvironmentControlled Environment
Easy to “segment” image
![Page 24: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/24.jpg)
241
Measurements from Measurements from ImagesImages
• Must have relationship between the image “pixels” and the world
• 2D imaging
– the image plane and the “world” plane are in 1-1 correspondence
• 3D –harder
![Page 25: Introduction to Machine Vision Systems](https://reader033.vdocuments.mx/reader033/viewer/2022061418/5681394e550346895da0e9c8/html5/thumbnails/25.jpg)
251
Goals for ME 439 and ME 739Goals for ME 439 and ME 739
• Modeling Cameras
– Basic of pinhole
• Kinematics of Vision
– Coordinate transformations
• Processing Images
– Some simple features (sections 8.13 - 8.25)
• 2D problems
• Modeling Cameras
– Pinhole model
– Projective mapping
– Calibration Procedures
• Kinematics of Vision
– Coordinate transformations
– Motion field equations
• Processing Images
– Feature detection (lines, blobs)
• Visual Servoing (Eye-Hand Coordination) in 3D