Slide 1

Introduction to NI Vision in LabVIEW for EducationGetting Started with Image Acquisition and Processing

ni.com#ni.comWelcome to NI Vision in LabVIEW for Education. This presentation will show you the powerful vision acquisition and processing tools available in LabVIEW for Education.1AgendaHistory of NI VisionVision Development ModuleAcquisitionImage ProcessingVision Builder for Automated Inspection (VBAI)Additional Resources

#ni.comToday we will cover a little bit of history of NI Vision in industry and research.Then show various ways of connect your camera to LabVIEW.Well take a look at both image acquisition and processing with the Vision Development Module in LabVIEW.And well explore Vision Builder for automated inspection, which is separate from LabVIEW.And finally, well review resources online and in the software that can help keep you going.2

NI Vision in IndustryNI Vision ApplicationsPart InspectionRobotic VisionQuality MeasurementsCrash Test AnalysisMedical Research

Browse Vision Case StudiesNI.com/Vision#ni.comFor over 15 years, National Instruments vision products have been helping engineers and scientists meet a diverse set of challenges faster and at a lower cost. National Instruments provides hardware and software tools for image acquisition and processing to address applications such as quality and process control, automated testing for semiconductor, automotive and electronics, intelligent monitoring, and medical imaging. In the picture you see here, the camera in the back looks at each bottle as it moves along the conveyer belt and automatically verifies that the bottle is filled to the correct amount, If it is not it can move actuators to move the bottle to a reject line.

Check out a wide variety of real world vision case studies to see exactly how this technology is used in industry and research today. http://www.ni.com/vision/applications/3NI Vision Industry Hardware

#ni.comNI Vision hardware provides a variety of options for different application needs, all powered by the same flexible software. Many engineers and scientists who want to include vision data with their measurements prefer the high speeds and synchronization options of plug-in NI frame grabbers. NI offers plug-in boards that work with Camera Link (the highest bandwidth currently available in a standard camera interface), IEEE 1394 (firewire) , GigE Vision, and parallel digital buses. For PC buses, NI vision hardware supports PCI Express, PCI, and PXI, PXIe, and USB.4Vision in LabVIEW for Education

LabVIEW

NI Vision Software

Hardware Drivers

#ni.comWhile NIs industry customers having been using vision software for over 15 years, in 2011 we decided to include our complete vision acquisition and processing software in LabVIEW for Education, the version of LabVIEW customized for high schools. This software gives students unprecedented access to powerful image processing tools that can be used well beyond the classroom.

LabVIEW for Education includesLabVIEW 2010 Service Pack 1NI Vision Software NI Vision Software includes 2 main application interfaces (VDM, VBAI) and the USB camera hardware Driver (IMAQdx 3.8)+ Other camera drivers(IMAQ, IMAQ I/O).Other non-vision hardware Drivers - myDAQ (DAQmx 9.2.3), LEGO NXT (Phantom) , LabQuest/SensorDAQ/Go!(Vernier Driver), fischertechnik ROBO TX5

NI Vision Camera Support

USB Cameras must be Direct-Show CompliantIs My Camera Supported?Troubleshooting USB CamerasWhat USB Cameras are Supported?

#ni.comBecause NI Vision was originally designed for industry and research, it supports hundreds of different kinds of cameras with a wide variety of connections. Many of these formats require special hardware, but NI Vision also support most low-cost USB webcams. In order to connect a USB webcam with any NI Vision software, the camera driver must be Direct Show compliant. If the driver is Direct Show Compliant the IMAQdx driver from NI will be able to connect it to NI Software. Generally speaking, USB 3.0 cameras will have the fastest frame rates and the most supported camera modes.

What USB Cameras are Supported?http://digital.ni.com/public.nsf/allkb/89E33B77CFE244E986256F8E004E0F13?OpenDocumentIs my Camera Supported? a full listing of all cameras supported by NI softwarehttp://digital.ni.com/public.nsf/allkb/4331B4AA3AE95233862574C200539A2F?OpenDocumentTroubleshooting USB Camerashttp://digital.ni.com/public.nsf/allkb/C1BC2D93ADC392BB8625768C0060AFE7?OpenDocument

6Vision Acquisition Express VI

Functions Palette Vision and Motion Vision Express

#ni.comFirst lets take a look at the Vision Acquisition Express VI. This function will allow you to connect to your USB Webcam and read in images by just configuring a few options in the configuration menu. You can find this Express VI on the functions palette in Vision and Motion Vision Express

The configuration menu will open when you create a new express VI or double-click on the icon.7

Vision Acquisition Express VISelect cameraClick snapshot or playClick Next

#ni.comFirst select a valid camera from the IMAQdx devices list. If you dont see your camera in this menu, try upgrading your webcam driver, or check out our troubleshooting guide. Press the snapshot or play button to test the connection. Once your camera is connected press next.

If your camera does not show up in this list, check the Troubleshooting USB Cameras Guidehttp://digital.ni.com/public.nsf/allkb/C1BC2D93ADC392BB8625768C0060AFE7?OpenDocument

8Vision Acquisition Express VI

Select acquisition modeClick Next

#ni.comThe Vision Acquisition Express VI needs to be configured for the acquisition mode. This effects how many images LabVIEW will read at a time and how it will store them in them temporarily. Lets look into these choices in more detail on the next slide.9Types of Image AcquisitionSingle Acquisition with Processing

Continuous Acquisition with Processing

Finite Acquisition with Processing

Finite Acquisition with Post-Processing

#ni.comWhen using any Vision Development Module in LabVIEW you should consider what type of acquisition you want to use.

Single Acquisition with Processing [Read, Process]This is useful for one shot processing, like count how many coins are on a table or, what color is on the camera

Continuous Acquisition with Processing [Read, Process, Repeat]This can be use to track something overtime like the position of an object or reading a digital clock display.

Finite Acquisition with Processing [Read X Images, Process as you go]This can be useful for analyzing sets of data for differences, like the difference in position of an object between frames (speed)

Finite Acquisition with Post-Processing [Read X Images Process after all images are ready]If you processing algorithm is very computer intensive and cant run while you read your images you can read them all at once, then process them after the fact.

For much more detailed information check out the Technical Primer on Image Acquisitionhttp://zone.ni.com/devzone/cda/tut/p/id/280810

Vision Acquisition Express VI

Choose video modeClick Next

#ni.comOnce you have connected to a camera you can select the mode to view images data in. Most USB cameras have several different images sizes and frame rates you can choose from. Smaller images work great for simple application and keep memory usage down, but you can go to a high resolution and fast frame rate to get more data. Remember small images will process much faster.11Vision Acquisition Express VI

Select the inputs and outputs you wantClick Finish

#ni.comFinally the last step in configuring the Vision Acquisition Express VI is selecting the inputs and outputs you would like to expose to the LabVIEW block diagram. Generally you always want to expose the Stop button, and Image Out, but you case decide if you need to show the image number, frame rate, or stopped indicator you can add them. For simple applications choose the fewest items to keep your code clean.12Vision Acquisition Express VI

IMAQ Image Display

#ni.comOnce you finish configuring the Vision Acquisition Express VI, you need to put it in a loop, add a stop button and an image indicator. (Control Palette Vision IMAQ Image Display)

Notice the controls and indicators show here, we did not choose to show the frame rate or image number. Note that on express Vis you can show/hide the terminals by shrinking the express block. (drag the blue handle on the top or bottom of the box)

It is also generally a good idea to use the error status to stop the loop in case the camera get disconnected. This code uses an or to stop the loop if there is an error OR if the acquisition is stopped.

For more examples using the Vision Acquisition express VI check outLabVIEW 2010\examples\IMAQ\IMAQdx Express Examples.llb (typically under C:/Program Files/National Instruments)

13IMAQdx Functions

Vision and MotionNI-IMAQdx

#ni.comThe Vision Acquisition Express VI provides an easy way to get started capturing images, but it has some limitations on speed and memory efficiency, so there are some times when using the low level IMAQdx functions will improve program performance. You can find these functions on the NI-IMAQdx subVIs under Vision and Motion on the functions palette14IMAQdx Example

LabVIEW 2010\examples\IMAQ\IMAQdx Examples.llb

#ni.comThis is a simple example that shows how to use the IMAQdx subVIs to continuously read images from the camera. The Low Level programming interface allows you to have much more control over speed and efficiency, but can require more detailed programming. This a very simple example that continuously displays images from a camera.

If you are just getting started with NI Vision you should get comfortable with the express Vis before moving onto the IMAQdx functions. You can find more examples at C:\Program Files (x86)\National Instruments\LabVIEW 2010\examples\IMAQ\IMAQdx Examples.llb

Steps:1. IMAQdx Open Camera - Open a connection to the camera, you must first select a Valid Session from the front panel.

2. IMAQ Create - This function created a spot in memory to put pictures. If we store each new image in memory we will soon run out. In this example we create 1 memory location named "myImage" and reuse it for each loop.

3. IMAQdx Configure Grab - This function tells the camera to start reading images so they will be ready when we ask for it.

4. IMAQdx Grab - Since we already started to camera in step 3, we just need to go and get the latest image from the camera then display it on the front panel.

5. IMAQdx Close Camera - We need to tell LabVIEW we are done using the camera so other programs can use it.

ProTIP: Express VIs automatically generate low level LabVIEW code, to make things easier for simple applications, you can actually see this code yourself. Right-Click on an Express VI and select Open Front Panel. You will be asked to convert the VI, if you select convert, you can then see all of the IMAQdx code behind an express VI. Be sure to configure before converting, once you convert you cannot convert back, just recreate the express VI if needed.15NI Vision SoftwareVision Development ModuleVision BuilderFor Automated Inspection

LabVIEW Add-On

Separate Application

#ni.comNI offers two distinct software packages tailored to the needs of programmers and manufacturing line operators, which include world-class image processing algorithms and image acquisition functions. With both options, you can make complex pass/fail decisions, customize user interfaces, and integrate with automation devices.

Vision Development Module is a LabVIEW Add-On that installs all of the subVIs and functions needed to acquire images from a camera and do analysis on them.

Vision Builder for Automated Inspection is a separate application (not part of LabVIEW) that allows you to use a configuration based interface (IE dropdown menus instead of code). It requires no programming.

The third component of NI Vision Software (not pictured) is the Vision Acquisition Software- a set of hardware drivers (including IMAQdx) that allows NI software to communicate with camera hardware. For most cases students will not need to interact with the driver, but it is required to connect wither VBAI or VDM to camera hardware.

More details-->http://digital.ni.com/public.nsf/allkb/45A5682F377996BB862572B50072787F

Note: NI Vision Software is only supported in Windows.16

LabVIEW Add-OnVision Acquisition and Processing in LabVIEWVision Development Module#ni.com17

Vision Development Module

VDM is a LabVIEW Add-On that includes specialized LabVIEW subVI (code) designed for image processing and analysis.

#ni.comThe Vision Development Module (VDM)providesmachine vision& image processing functions (subVIs) for LabVIEW.In general, VDM functionsare to be used for post processing images. Some examplefunctions are: pattern matching, particle analysis, edge detection, thresholding, histograms, and optical character recognition. In addition,VDM includesVision Assistant, a prototyping and code generation tool similar to VBAI in its menu-driven interface. Vision Assistant is a useful tool for developing and testing a series of image processing steps from which code can then be generated. VDM is not intended for any specific industry or application. Rather it is part of the LabVIEW programming language and is used to build customized vision applications.

This picture shows a Vision Acquisition Express VI which is part of the IMAQdx driver, and a Vision Assistant image processing express VI that we will learn more about later. This VI is designed to read in an image, process it and display the results continuously.

18VDM Image Processing FunctionsMatch PatternMatch Geometric PatternDetect ShapesEdge DetectorPeak Valley DetectorCaliper ToolLine Gauge ToolRakeConcentric RakeSpokeStraight EdgeClampMeasure IntensityDetect ObjectsMotion EstimationOCR (Read and Grade)ClassificationColor SegmentationDetect Texture DefectContour AnalysisGolden Template Comparison7 Segment LCD ReaderBarcode Readers (1D, QR, DataMatrix, PDF417)DataMatrix Barcode GradingRead Dial MeterGeometry (Get Mid Point, Distance, Angle, Best Fit, Intersection, Perpendicular Projection, Bisecting Line, etc.)Frequency MeasurementsOperators (Add, Subtract, Multiple, Divide, AND, OR, XOR, Compare, etc

Browse Full Function List

#ni.comNow that you can read in an image how are we going to process it. Vision Development module includes dozens of the most common image processing routines and algorithms. (~154 functions)

Browse NI Vision Function Listhttp://zone.ni.com/reference/en-XX/help/370281M-01/nivisionlvbasics/ni_vision_function_palettes/19Vision Assistant Express VI

#ni.comThe easiest way to get started processing is using the Vision Assistant. Just like the acquisition express VI it allows you to configure a custom processing step and plug it into an existing LabVIEW program. Double click the express VI to open the configuration menu.20

Vision Assistant Express VIOriginal ImageCurrent Step ImageFunctionMenuProcessing ScriptConfigMenu

#ni.comThe vision assistant window allows you to configure and test your custom vision processing algorithm using simple configuration options.

In the upper right corner you see the original image as captured by your camera, when you start the Vision Assistant it will grab a current image so go ahead and make sure its pointing at your subject. In the low right corner, you can see the function menu. Browse between basic sets of functions to break an image up and process the data. The Large window shows the image proceed by the current step. You can click through the steps on in your script and see the results of each step.The bottom of the screen shows the processing script, the set and order of processing functions to use on the image, You can double click each item to configure it.When you select a specific step the configuration menu shows the properties you can adjust.

21Vision Processing

Segment/simplify image2.Clean up selection3.Extract featuresOutput

#ni.comThis shows the general way most vision processing algorithms should work. In this case lets look at an example where we want to track the position of a pocket watch pendulum. So we will need to find the X,Y coordinates of the watch

Step 1. Segment or simplify the image. Images contain a lot of information, and processing a full color image is not needed or even helpful in most cases, so step 1 is to simplify your image somehow. This this case we are going to make a binary image (two colors) and pick the a segment of the RGB planes so that we can find the pocket watch.You might Convert to Grey scale, Select a single color plane (R/G/B), Select a brightness or saturation level, or a wide multitude of options.

Step2. Clean Up the SelectionOften the setting may leave small mistakes or artifacts that will mess up your calculations. In this case we need to remove the red selection along the edge and clean a few small dots around the screen. There are a wide variety of functions you can use to clean up your selection, be sure to try with different images under different lighting conditions to build a reliable system.

Step 3. Extract FeaturesNow that we have a nice clean blob on a binary image we can use some functions to get information about it. In this case we use the Particle Analysis step to find the bounds of the red blob and the X and Y coordinates of the center. We can select to return that data to LabVIEW.

This basic method will work for most cases weather you are tracking a color, and object, or making measurements.

22

Vision Assistant ExampleSegmentCleanExtract

#ni.comHere is the vision assistant used in the previous example, lets scroll through and see what the different steps look like.23

Vision Assistant Express VI

#ni.comOnce you have configure the Vision Express VI you can use it in conjunction with the Vision Acquisition VI to do live image processing in LabVIEW.24VDM Functions

#ni.comWhile the Vision Assistant Express VI can do most common functions there are cases when you need more control or more interaction with your data. Using the Vision Development Module processing and machine vision functions you can expand way beyond the capability of the Vision Assistant.

The Image Processing palette allows you to perform mathematical operations on the image data, you can pick out a specific color, take an average, or analyze the size of a blob and much more.

The machine Vision Palette is comprised on subVIs specifically designed for machine vision inspection, you can measure distances, compare templates, read text and classify objects and much more.

The Vision Utilities provide neccessary functions to manage memory, manipulate image data and overlay the results and much more.25

Vision Processing

#ni.comHere is an example of a block of Vision analysis code using the VDM subVIs. To find a full library of examples, in LabVIEW go to Tools Find Examples and on the browse tab select Toolkits and Modules Vision and browse though application specific examples and examples for most functions.26Vision Builderfor Automated InspectionVision Acquisition and Processing outside LabVIEWStand Alone Application

#ni.com27

Vision Builderfor Automated Inspection

VBAI Interactive TourVBAI Data Sheet (Basic Info)#ni.comNational Instruments Vision Builder for Automated Inspection (Vision Builder AI) is configurable software for building, benchmarking, and deploying machine vision applications. NI Vision Builder AI does not require programming. VBAI was designed for use in an industrial setting in which repetitive inspections are needed to verify manufacturing processes. A built-in deployment interface is included so you can quickly deploy your inspection, guidance, and identification applications. Notice that VBAI looks very similar to the Vision Assistant, with two main differences, VBAI does Acquisition AND Processing, and VBAI exists completely separate from LabVIEW.

VBAI DataSheet (Basic Info)http://www.ni.com/pdf/products/us/vbai_datasheet.pdfVBAI Interactive Tourhttps://ni.adobeconnect.com/_a56821929/p17144858/VBAI Tutorialhttp://www.ni.com/pdf/manuals/373379h.pdf

28

Vision Builderfor Automated Inspection

Main WindowOverview Window State Configuration Window Inspection Steps Palette #ni.comMain windowDisplays the image being processed, property pages for some inspection steps, or the state diagram for the inspection. Use the Main window to define regions of interest in an image, configure step parameters for some steps, and create/modify the state diagram for an inspection. Overview windowDisplays a thumbnail view of either the current inspection image, or the state diagram for the inspection. Inspection Steps paletteLists and describes the steps that you use to create your inspection. When you click on a step, this palette transforms into the property page for the step. State Configuration windowDisplays the list of steps that comprise the currently selected state in the inspection.

Press the Toggle Main Window to swap the state diagram and the current inspection image.

29

Vision Builderfor Automated Inspection

#ni.comThe state Inspection diagram allows you to visualize the steps in your inspection process as a state chart. By selecting the state you can edit or run each part of a larger inspection. 30

VBAI TutorialVBAI Example

#ni.comIn this example VBAI is used to automatically inspect a chocolate chip cookie to verify it was made correctly. In this inspection we read in the image, segment the color to mark the different sections, calculate some numbers (% cooked and % chocolate exposed) if these numbers are within the set limits the cookie will be verified as tested otherwise it will fail. Inspections do not only apply to mechanically parts, but food, drinks, animals, traffic, even people or just about anything else.

To open and explore this example and many others browse to National Instruments\Vision Builder AI 2010\Examples(Typically in C:\Program Files (x86))This example is called Segment Color Image Example.vbai

31Vision Builderfor Automated Inspection

#ni.comVision Builder only has a limited selection of the most common functions that in the VDM, but now without having to write any code you can set up a custom vision processing inspection and run it without opening LabVIEW. 32VBAI Example

#ni.comLets take a look at a few more examples of what kind of image processing can be done in VBAI.

All of the examples on this slide can be found in National Instruments\Vision Builder AI 2010\Examples(Typically in C:\Program Files (x86))

33VBAI LabVIEW Integration

VBAI call a custom LabVIEW subVILabVIEW can run a VBAI InspectionVBAI/ LabVIEWInteractionMigrate Code#ni.comAlthough VBAI and LabVIEW (with VDM) are separate applications they can interact. If you have already created an inspection script in VBAI and want to integrate it with a larger LabVIEW application, you can call the inspection in VBAI, run it, then get the results by using the VBAI palette for LabVIEW.

In VBAI you can also go to ToolsMigrate Inspection to LabVIEW and VBAI will automatically generate LabVIEW code that is equivalent to your VBAI inspection. Because this code is automatically generated it can be more challenging to read. Creating a inspection in VBAI then migrating it to LabVIEW is another way of getting started in VDM. VBAI Also allows you to export an inspection to C code so you can integrate it into C based programs.

If you have a VBAI inspection and want to add some myDAQ functionality or some other custom LabVIEW behavior, you can call a custom subVI from VBAI. For instance you could create a subVI that connects to myDAQ and turns an LED On. If your VBAI test was successful you could call that subVI and turn on the light.

More info on VBAI/LabVIEW Integrationhttp://zone.ni.com/devzone/cda/pub/p/id/124234NI Vision ResourcesMemory Management with IMAQ IMAQ Error GuideIs My Camera Supported?Technical Primer on Image AcquisitionA Practical Guide to Machine Vision LightingTroubleshooting USB CamerasWhat USB Cameras are Supported?VBAI Interactive TourVBAI TutorialVBAI DataSheet (Basic Info)VDMWebcast Series(LabVIEW)ToolsHelpNI Vision for LabVIEW HelpBrowse Full Function List

#ni.comHere are links related to NI Vision that can help get you started and help you troubleshoot any issue.

Memory Management with IMAQ - Advanced tips for making programs efficient (and fast)fhttp://zone.ni.com/devzone/cda/tut/p/id/4538Where Can I Find Details on IMAQ or Vision Error Codes?http://digital.ni.com/public.nsf/websearch/C3C20E427DE91928862570DE00542B7D?OpenDocumentIs my Camera Supported?http://digital.ni.com/public.nsf/allkb/4331B4AA3AE95233862574C200539A2F?OpenDocumentTechnical Primer on Image Acquisitionhttp://zone.ni.com/devzone/cda/tut/p/id/2808A Practical Guide to Machine Vision Lightinghttp://www.graftek.com/pdf/Marketing/MachineVisionLighting.pdfTroubleshooting USB Camerashttp://digital.ni.com/public.nsf/allkb/C1BC2D93ADC392BB8625768C0060AFE7?OpenDocumentWhat USB Cameras are Supported?http://digital.ni.com/public.nsf/allkb/89E33B77CFE244E986256F8E004E0F13?OpenDocumentVBAI DataSheet (Basic Info)http://www.ni.com/pdf/products/us/vbai_datasheet.pdfVBAI Interactive Tourhttps://ni.adobeconnect.com/_a56821929/p17144858/VBAI Tutorialhttp://www.ni.com/pdf/manuals/373379h.pdfBrowse NI Vision Function Listhttp://zone.ni.com/reference/en-XX/help/370281M-01/nivisionlvbasics/ni_vision_function_palettes/VDM Webcast Serieshttp://www.ni.com/visionwebcastseries/

35ni.com/stem-education

#ni.com