T1 - The Fundamentals of Machine Vision AND OVERVIEW The Fundamentals of Machine Vision – What is Machine Vision – The Machine Vision Market – Industrial Uses of Machine ...

Download T1 - The Fundamentals of Machine Vision  AND OVERVIEW The Fundamentals of Machine Vision – What is Machine Vision – The Machine Vision Market – Industrial Uses of Machine ...

Post on 06-Feb-2018

226 views

Category:

Documents

9 download

TRANSCRIPT

  • TheFundamentalsofMachineVision

    DavidDechowStaffEngineer,IntelligentRobotics/MachineVision

    FANUCAmericaCorporation

  • INTRODUCTIONANDOVERVIEWTheFundamentalsofMachineVision

    WhatisMachineVision TheMachineVisionMarket IndustrialUsesofMachineVision

  • IntroductionandOverview WhatisMachineVision

    Machinevisionisthesubstitutionofthehumanvisualsenseandjudgmentcapabilitieswithavideocameraandcomputertoperformaninspectiontask.Itistheautomaticacquisitionandanalysisofimagestoobtaindesireddataforcontrollingorevaluatingaspecificpartoractivity.

    KeyPoints: Automated/NonContact Acquisition Analysis Data

  • IntroductionandOverview WhatisMachineVision

    ImageAcquisition

    SensorsOptics

    Lighting

    ImageAcquisition

    Analysis

    ComponentsSoftware

    AlgorithmsAnalysis

    Integration and

    Applications

    ResultsCommunications

    Automation

    IntegrationandApplications

  • IntroductionandOverview TheMachineVisionMarket

    Choices Wellover400manufacturersandsuppliers Diverseproductofferings

    Confusion Product/componentdifferentiationsometimesisunclear Endusers(thebuyers)oftendontunderstandwhattheyaregetting

    Whatsimportant Componentsandtechniquesneedtobebetterunderstoodattheenduserlevel

    Advancedtechnologyskillsarenecessaryforcompetentspecificationandintegration

  • IntroductionandOverview TheMachineVisionMarket

    GeneralPurposeMachineVisionSystems

    PCbasedsystem Singleormultiplecamerasinterfacedtoacomputer,standard(Windows,Linux)

    operatingsystem Diverseimagingdevicesavailable

    analog(RS170),anddigital(GigEVision,FireWire,CameraLink,USB)interfaces

    CameraLens

    ImagerElectronics

    Power/ControlSignal

    FrameGrabberorothersignalconversion

    Computer

    DigitalImage

  • IntroductionandOverview

    TheMachineVisionMarket GeneralPurposeMachineVisionSystems,continued

    Camerasensorandproprietarycomputerinonepackage,proprietaryoperatingsystem,ethernet communications

    Applicationconfigurationexternaltothedevice

    CameraLens

    ImagerElectronics

    Power/ControlSignal

    Computer

    Optionalext.computerforoperatorinterface

    DigitalImage

    FrameGrabberorothersignalconversion

    VisionSystemPackageAllinOneSystemSmartCameraVisionSensor

  • IntroductionandOverview

    TheMachineVisionMarket GeneralPurposeMachineVisionSystems,continued

    Camerasensorormultipletetheredcameraswithfullcomputer(keyboard,mouse,monitor,USB,Ethernet),standard(Windows,Linux)orproprietaryoperatingsystem,

    CameraLens

    ImagerElectronics

    Power/ControlSignal

    Computer

    DigitalImage

    FrameGrabberorothersignalconversion

    VisionSystemPackageHybridSmartCamera

  • IntroductionandOverview TheMachineVisionMarket

    Systemfeatureoverview PCbasedsystems

    Mostflexibleandpowerfulsystemdesign Degreeofdifficultyvariesbyimplementation Pricingvariesdependinguponarchitecture

    SmartCamera/SmartSensorvisionsystem Includestheeasiesttousesystems

    Somearemoredifficulttouse Greaterdangerofoverspecifyingcapability Pricingvarieswidely canbequiteinexpensive

    HybridSmartCameravisionsystem Includessomeofthefeaturesofbothdependinguponproduct Somearchitecturesmayposeintegrationchallenges

  • IntroductionandOverview TheMachineVisionMarket

    Camera/processorhardwareisjustanimagedeliverysystem!! Differentiationofproductsatthehardwarelevelislimitedto:

    Physicalstructureandsystemarchitecture Singleormultipleviews?

    Smartcamera distributedsystem PCbased centralizedsystem

    Customorfixedinterfaceoptions Availablecameraresolutions Processingspeeds Input/outputoptions Otherhardwareintegrationissues

  • IntroductionandOverview TheMachineVisionMarket

    Peripheralcomponents Lighting Optics I/Odevices FrameGrabbers

  • IntroductionandOverview

    TheMachineVisionMarket ApplicationSpecificMachine

    VisionSolutions(ASMV) Standalonedevicesdesignedfortargetedinspectiontasks

    Imaging,lighting,optics,automation

    Benefitisagenerallyuncomplicatedandeasytouseinspectiondeviceforafocusedapplicationarea

  • IntroductionandOverview TheMachineVisionMarket

    Targetedapplicationcomponents Bar and2D codereaders Othersmartsensors

    ProductimagescopyrightCognex,Siemens

  • IntroductionandOverview IndustrialUsesofMachineVision

    UsersofMachineVisionbyIndustry

    0Wood

    Semiconductor

    Electronics/Electrical

    Automotive

    Pharma/MedicalDevice

  • IntroductionandOverview Industrialusersofmachinevision

    Agriculture,Automotive,Biometrics/Security,Container,Cosmetic,Electronics/Electrical,Entertainment,FabricatedMetal,Fastener,Food/Beverage,Glass,LabAutomation,Lumber/Wood,MedicalDevices,MedicalImaging,Military/Aerospace,MiscellaneousMfg.,Nanotechnology,Paper,Pharmaceutical,Plastics,PrimaryMetal,Printing,Rubber,ScientificImaging,Semiconductor,Telecommunications,Textile/Apparel,Tobacco,Transportation

  • IntroductionandOverview IndustrialUsesofMachineVision

    Machinevisionapplicationcategories Defectdetection Gauging Guidanceandparttracking Identification OCR/OCV Packaginginspection PatternRecognition ProductInspection SurfaceInspection WebInspection

  • IMAGEACQUISITIONTheFundamentalsofMachineVision

    Sensors&Imaging Optics Lighting

    Acquisition

    Analysis

    Integration

  • ImageAcquisition Nothinghappensinamachinevisionapplicationwithoutthe

    successfulcaptureofaveryhighqualityimage Imagequality:correctresolutionforthetargetapplicationwith

    bestpossiblefeaturecontrast Resolution determinedbysensorsizeandqualityofoptics Featurecontrast determinedbycorrectlightingtechniqueandqualityofoptics

    Imagingissaidtocontributemorethan85%tothesuccessofanymachinevisionapplication

    Thegoalofmachinevisionimageacquisitionistocreateanimagethatisusablebythetechnology notnecessarilyonethatspleasingtothehumaneye

  • ImageAcquisition

    SensorsandImaging Allmachinevisioncameras

    createanimagebyexposingarraysofphotosensitivematerialtolightenergy

    Thinkofphotonbuckets Exposuredurationistimelimitedandtypicallyadjustable

    Theenergyinabucketcapturedduringanexposureperiodbecomesamicrovoltageforthatbucket

  • ImageAcquisition

    SensorsandImaging Eachelementinacamera

    sensorarrayiscalledapixel(pictureelement)

    Theenergyvalueforeachindividualpixelisoutputasamicrovoltageuponacquisitionofeachimage thevoltageultimatelydeterminesthecolorlevelforthatpixel

    Thepixelanddatatransferarchitecturevariesbysensortype

    mostwidelyusedareCCDandCMOS

  • ImageAcquisition SensorsandImaging

    Theimagingsensorarraycomesindifferentphysicallayouts Area Line

    Sizeofthechipvarieswidelyasdoesthenumberofindividualpictureelements(pixels)

    Typicalareachipformachinevision:from.3to5+Mpix 640to2048+pixels(horizontal) Physicalsizesfromdiag.upto1+

    Typicallinescanarray:from1Kto12K+pixels Physicalsizesfromabout15mmto90mm+

  • ImageAcquisition SensorsandImaging

    Imagerepresentationinthecomputer

    255 255 255 105 51 41 43 49 101 255 255 255

    255 255 255 116 62 44 42 57 120 255 255 255

    255 255 255 112 68 41 46 58 117 255 255 255

    105 110 111 109 60 42 48 61 115 112 114 108

    60 68 62 57 42 41 46 41 43 49 42 41

    44 42 41 46 46 42 48 44 42 42 46 42

    41 46 42 48 44 42 41 41 46 43 49 42

    59 54 60 59 41 46 42 46 46 42 48 46

    100 120 120 115 51 41 43 49 110 116 118 105

    255 255 255 118 62 44 42 57 115 255 255 255

    255 255 255 121 68 41 46 58 120 255 255 255

    255 255 255 100 60 42 48 61 105 255 255 255

  • ImageAcquisition SensorsandImaging

  • ImageAcquisition SensorsandImaging

    Whataboutcolor? Bayerfilter Threechip

  • ImageAcquisition SensorsandImaging

    Imageresolution Keyelementinvisioncomponentselection Thesmallestfeatureresolvedbytheimager Determinants:Whatisthesizeofthefieldofview(FOV),andwhatistherequiredaccuracyoftheimaging.

    Howmanypixels?(forexampleonlyactualrequirementvariesbyapplication)

    Inspection Pixelsrequired(usually)

    Defectdetection Min.2x2

    Featurelocation Min. 3x3

    Featuredifferentiation Min.5x5

    Gauging Subpixel resolutionmustbe1/10th thedesiredtolerance

  • ImageAcquisition

    SensorsandImaging ImageResolution

    ExamplesWeneedtodetecta0.1(diam.)defect(highcontrast)onasurfacethatis3square.Givengoodlightingandhighqualityoptics,whatcameraresolutiondoweneed?

    Thedefectdiametershouldspanabout2pixelssoapixelmustcover0.05.Over36therefore,theremustbe720pixels(36/0.05).Wemustselectacamerawithatleastthatresolutionintheminoraxis(vertical) probablyonewitha1024x780sensor.

    Wemustdifferentiateanemblemthatisapproximately1highrelativetoaverysimilarfeatureinalowcontrastimage.Ifweuseastandardresolutioncamera(640x480),howlargeshouldthefieldofviewbe?

    Atminimum,adifferentiableobjectmustcover5pixels.Duetothelowcontrast,wedecidetodoublethatcoverageto10pixels.Thetargetpixelsizewillbe0.1(1/10),andthefieldofviewmustbenolargerthan48(480x.01).

  • ImageAcquisition Optics

    Applicationofopticalcomponents Machinevisionrequiresfundamentalunderstandingofthephysicsoflensdesignandperformance

    Goal:specifythecorrectlens Createadesiredfieldofview(FOV) Achieveaspecificoracceptableworkingdistance(WD) Projecttheimageonaselectedsensorbasedonsensorsize primarymagnification(PMAG)

    Createthehighestlevelofcontrastbetweenfeaturesofinterestandthesurroundingbackground;withthegreatestpossibleimagingaccuracy

  • ImageAcquisition Optics

    Considerationsforlensselection Magnification,focallength,depthoffocus(DOF),fnumber,resolution,diffractionlimits,

    aberrations(rolloff,chromatic,spherical,fieldcurvature,distortion),parallax,imagesize,etc. Somegeometricaberrationmaybecorrectedincalibration

    Thephysicsofopticaldesigniswellknownandcanbemathematicallymodeledand/orempiricallytested

    Specificationorcontrolofmostofthelenscriteriaisoutofourhands

    S5 S6 Images:EdmundOptics;www.edmundoptics.com

  • ImageAcquisition

    Optics Considerationsforlensselection

    Practicalspecificationsformachinevision:PMAG(asdictatedbyfocallength)andWDtoachieveadesiredFOV

    Useasimplelenscalculatorand/ormanufacturerlensspecifications

    Simple statetherequiredFOV,thesensorsizebasedonphysicalselectionofcameraandresolution,andadesiredworkingdistancecalculatethelensfocallength

    Note specifiedworkingdistancemaynotbeavailableforagivenlens reviewspecifications

    Testyourresults Alwaysuseahighresolutionmachine

    visionlensNOTasecuritylens

    Images:PPTVision;pptvision.com

  • ImageAcquisition

    Optics Whyusemachinevisionlensesonly

    Lightgatheringcapabilityandresolution

    Images:EdmundOptics;www.edmundoptics.com

  • ImageAcquisition Optics

    SpecialtyLenses Telecentric Microscopestages Macro,longWD Zoom(cautionrecommended)

    Images:EdmundOptics;www.edmundoptics.com,Navitar;www.navitar.com

  • ImageAcquisition Lighting

    Scienceorart??? Correctlightingmust

    Highlightfeaturestobedetectedrelativetobackground Createrepeatableimagesregardlessofpartvariation

    Incorrectlightingwillputthesuccess/reliability/repeatability/easeofuseofthevisionapplicationatrisk

    MachinevisioncamerasandsoftwarealgorithmsCANNOTmakeupforinadequateilluminationtechniques

  • ImageAcquisition

    Lighting Illuminationformachinevisionmust

    bedesignedforimaging,nothumanviewing

    Selectionmustbemaderelativetolightstructure,position,color,diffusion

    Weneedtoknowhowlightworkssoourlightselectionsarenothitandmissguesswork

    Lightisbothabsorbedandreflectedtosomedegreefromallsurfaces

    Whenanobjectisclearortranslucent,lightisalsotransmitted

    Angleofincidence=angleofreflection

    LightSource SpecularReflection

    TransmittedLight(ifobjectisnotcompletelyopaque)

    DiffuseReflection

    Refraction,Absorption

  • ImageAcquisition Lighting

    Dedicatedlightingmustbeusedformachinevisionwithfewexceptions.

    Wherefeasible,LEDilluminationisthebestsource Longlifewithminimaldegradationofintensity Abletobestructuredintoavarietyofshapes

    Maybedirectionalordiffuse Maybestrobedatveryhighdutycyclesandoverdriventomanytimesnominalcurrentspecifications

    Availableinmanyvisibleandnonvisiblecolors Othersources fluorescent,fiberoptics

    Fluorescent brightandhighlydiffusebutcanbeinconsistent Fiberoptic glass/plasticfibersdeliveringlightfromhalogen,tungstenhalogenorxenonsource,bright,shapeable,focused

  • ImageAcquisition Lighting

    LightingTechniques Thegoaloflightingformachinevisionapplicationsusuallyistomaximizethecontrast(grayscaledifference)betweenfeaturesofinterestandsurroundingbackground

    Techniquesarecategorizedgenerallybythedirectionoftheilluminationsource

    Mostmaybeachievedwithdifferentsources

  • ImageAcquisition

    LightingTechniques Directbrightfield

    illumination Sources:highangleringlights(shown),spotlights,barlights(shown);LEDsorFiberopticguides

    Uses:generalilluminationofrelativelyhighcontrastobjects;lightreflectiontocameraismostlyspecular

    Images:CCSAmerica;www.ccsamerica.com

  • ImageAcquisition

    LightingTechniques Diffusebrightfield

    illumination Sources:highanglediffuseringlights(shown),diffusebarlights;LEDsorfluorescent

    Uses:generalilluminationofrelativelyhighcontrastobjects;lightreflectiontocameraismostlydiffuse

    Images:CCSAmerica;www.ccsamerica.com

  • ImageAcquisition

    LightingTechniques Directdarkfieldillumination

    Sources:lowangleringlights(shown),spotlights,barlights;LEDsorFiberopticguides

    Uses:illuminationofgeometricsurfacefeatures;lightreflectiontocameraismostlyspecular

    Darkfieldismisleadingthefieldorbackgroundmaybelightrelativetosurfaceobjects

    Images:CCSAmerica;www.ccsamerica.com

  • ImageAcquisition

    LightingTechniques Diffusedarkfieldillumination

    Sources:diffuse,lowangleringlights(shown),spotlights,barlights;LEDsorfluorescent

    Uses:nonspecularilluminationofsurfaces,reducingglare;mayhideunwantedsurfacefeatures

    Images:CCSAmerica;www.ccsamerica.com

  • ImageAcquisition

    LightingTechniques Diffusebacklight

    Sources:highlydiffusedLEDorfluorescentarealighting

    Uses:provideanaccuratesilhouetteofapart

    Images:CCSAmerica;www.ccsamerica.com

  • ImageAcquisition

    LightingTechniques Structuredlight

    Sources:FocusedLEDlineararray,focusedorpatternedlasers

    Uses:highlightgeometricshapes,createcontrastbaseduponshape,provide3Dinformationin2Dimages

    Images:CCSAmerica;www.ccsamerica.com,Stocker&Yale;www.stockeryale.com

  • ImageAcquisition

    LightingTechniques Onaxis(coaxial)illumination

    Sources:directed,diffusedLEDorfiberopticarea

    Uses:producemoreevenilluminationonspecularsurfaces,mayreducelowcontrastsurfacefeatures,mayhighlighthighcontrastgeometricsurfacefeaturesdependingonreflectiveangle

    Images:CCSAmerica;www.ccsamerica.com

  • ImageAcquisition

    LightingTechniques Collimatedillumination

    Sources:specialtyilluminator(LED,Fiber)utilizingopticstoguidethelight

    Uses:highlyaccuratebacklighting,reducingstraylight,highlightingsurfacefeaturesasafrontlight

    Images:EdmundOptics;www.edmundoptics.com,CCSAmerica;www.ccsamerica.com

    NondirectionalCollimated

  • ImageAcquisition

    LightingTechniques ConstantDiffuseIllumination

    (CDI cloudydayillumination)

    Sources:specialtyintegratedlighting

    Uses:providescompletelynonspecular,nonreflectingcontinuouslightingfromallreflectiveangles;goodforreflectiveorspecular surfaces

    Images:Siemens;www.nerlite.com

  • ImageAcquisition LightingTechniques

    Otherlightingconsiderations Color

    Monochromaticlightoncoloredfeatures

    Cameraresponsetodifferentcolors Whitelightandcolorimaging Nonvisiblecolors

    Lightdegradationovertime;componentlife,heatdissipation

    Lightintensityanduniformity Strobing Eliminationofambientandotherstraylight

  • IMAGEANALYSISTheFundamentalsofMachineVision

    MachineVisionSoftware GeneralMachineVisionAlgorithms

    Acquisition

    Analysis

    Integration

  • ImageAnalysis MachineVisionSoftware

    Machinevisionsoftwaredrivescomponentcapability,reliability,andusability

    Mainmachinevisioncomponentdifferentiationisinthesoftwareimplementation

    Availableimageprocessingandanaly...

Recommended

View more >