A 3-D view of vision tools
A plot of vision applications helps you choose appropriate inspection software.
Irene Leszkowicz and Sarah Sookman, Matrox Imaging -- Test & Measurement World, 8/1/2003
In machine-vision applications, using the right tools eases software development and produces a system that will last for a long time. To help narrow your choice of software tools, we recommend you use a three-dimensional (3-D) graph to plot the characteristics of your application. This graph can help you choose from four fundamental types of analysis software when setting up a machine-vision system.
All machine-vision applications have three general characteristics that affect the type of analysis tools you need. We represent the three characteristics on the axes of a 3-D graph (Figure 1):
- The shape of the object, and the number of objects in the field of view (y axis).
- The predictability of an object's position and orientation (z axis).
- The lighting used to illuminate an object (x axis).
You can use these axes to plot the characteristics of your application; the images in Figure 1 will help you decide where your application falls. Your application won't necessarily match a single point on each axis, so you can indicate a general range into which it fits. Once you've plotted your application, you can determine which types of software tools you need.
Y axis—How many?On the y axis, you plot the complexity of your application. "Complexity" encompasses the size, shape, and number of objects a system must inspect. A simple application may inspect one object, such as a switch or an IC. A more complex task could measure the characteristics of identical objects, say pins on a connector or ICs in a tray. Inspection of individual objects requires software that locates objects, analyzes them, and reports results.
The most complex inspection tasks include those that check PCBs for correct component orientation or solder quality. The software for PCB inspection, for example, must quickly analyze large numbers of small items.
Use the z axis to indicate whether your software tools will need to locate an object and determine its orientation. Semiconductor inspection generally falls near the origin on the z axis, because wafer-handling equipment provides precise placement with little angular variation. If you need to look at components on a PCB, though, machine-vision software must account for small changes in the board's position due to production variations. At the extreme, you may need to look for several types of components at any location and any angle.
X axis—Can you see it clearly?You locate your application along the x axis according to its lighting conditions—ranging from simple backlighting to complex lighting schemes that create shadows or specular highlights. Strong shadows cast by an object can cause software to "see" edges that don't exist, thus confusing some object-locator tools. And shadows cast across an object can obscure important features. Specular highlights—shiny reflective spots—can confuse some tools that use intensity differences to determine thresholds or identify edges.
When you design a vision system, you won't have much control over the objects under inspection, but you might be able to improve the lighting. Generally, if there's room within your system to mount more lights, and you can position them for improved illumination (to minimize shadows, for example), you should do so. Improved lighting is generally preferable to the adoption of a complex software scheme that attempts to deal with poor lighting (Ref. 1).
 |
|
Figure 1. The lighting axis, x, shows a range from simple backlighting that produces a black-and-white image to lighting that produces an image that includes reflections and "noise," as shown in the "T" brackets. The y axis represents the shape, size, and quantity of the objects you must inspect. The complexity of the images increases further from the origin. Points along the z axis let you plot the location and orientation complexity of an application. |
When selecting machine-vision software, you'll need to consider four types of tools. The first two—profile analysis and blob analysis—count objects and perform simple locating and measuring operations. The other two—normalized gray-scale correlation and edge-based geometric pattern matching—locate, count, and identify objects in situations where the first two tools cannot. (Although not covered here, many tools also can read bar codes, OCR character sets, and data-matrix codes for product tracking and identification.)
• Profile analysis. A profile-analysis tool finds an edge, provides its location and its angle with respect to a reference, and finds the distance between two edges. An application could use profile analysis to detect the absence of pins in a connector and to count the number of pins. Some profile-analysis tools can supply information such as whether the pins are parallel to each other. (Vendors may call profile analysis by names such as "measure across," "find line," edge, and stripe.)
A profile-analysis tool has a limitation: It must know in advance the path—a line or an arc—along which it will make measurements. It's simple to define a fixed path across a series of PCBs firmly held in fixtures on a production line, but it's more complex to define a profile path across a row of connector pins, if the connector can appear in an image at any angle. Machine-vision applications often use a separate tool to supply an object's position to a profile-analysis tool.
• Blob analysis. Blob analysis operates on images first subjected to thresholding, a process that constructs a binary (black-and-white) image from a gray-scale image. This process lets a blob-analysis tool identify touching pixels that belong to a group or object. An analysis can produce values such as size, area, perimeter, and minimum and maximum diameter. If you need to count multiple objects of non-uniform shape, a blob analysis provides the most flexibility, but it generally doesn't match images against a model of an ideal shape.
Some machine-vision software can compare blob measurements, known as "feature sets," with previously established "trained sets." This operation helps identify objects that have distinct features. A comparison could classify circular objects with a smooth edge and one hole as washers, and classify circular objects with a rough edge and one hole as lock washers.
• Normalized gray-scale correlation. A normalized gray-scale correlation (NGC) tool "scores" the similarity between a rectangular "model" image and the area around every pixel in a larger image. The highest scores indicate matches between objects in the image and the model. Normalization adjusts images that have lighter or darker illumination than the model image, so the correlation can operate on a wide range of image types. Don't confuse the NGC technique with techniques that simply subtract one image from a "golden image" to find differences.
A brute-force correlation requires many calculations, the number of which increases rapidly as the model size and the size of the region being searched increase. In "golden-template" matching, the template and the target will be almost identical, so a simplified correlation may suffice for alignment of the target with the template. Then, blob analysis can analyze any remaining clusters of pixels that are very different. Many algorithms seek to optimize these basic correlations, but efficiently searching for multiple instances of an object anywhere in a large region benefits from NGC tools.
Optimized NGC tools reduce processing times by identifying likely locations of objects and running searches in those areas. An NGC tool can match objects with subpixel accuracy, but it cannot tolerate size and angle variation: Objects must appear in the same position and have the same orientation as the object used for the model image. An NGC tool will quickly locate fiducial marks on boards and bonding pads on dies. Although the tool can reject individual "noisy" pixels, it cannot locate highly reflective or obscured objects.
• Edge-based geometric pattern matching.This technique compares sets of vectors obtained from an object with sets of vectors constructed for a known object. Pixels at the edges of the objects define the vectors. Even if something obscures part of an object, as long as about 60% or more of its edges remain visible, an edge-based pattern-matching tool will find it. Some software will calculate the location of the edges with subpixel accuracy and give good results in uneven lighting.
Because vectors describe only edges, algorithms can rotate and scale edge data to find an object, regardless of its orientation or size. The ability to handle a range of magnifications means multiple inspection stations do not need the repeatability required by an NGC tool. And a vector-based tool can account for distortion in an image, so you don't need to exactly duplicate in the factory the vision system that the developers used to define vectors for an object.
Choose your tool
 |
|
Figure 2. Overlaying the x, y, and z axes with the four fundamental machine-vision tools lets you plot your application at a point or small space on the 3-D plot. You can choose tools that coincide with your plotted needs. |
All the tools can handle applications that fall close to the origin. The tools can locate, count, and measure identical objects such as a row of connector pins or a grid of pixels in an LCD. Often, you need to do little or no work beyond setting a few values in dialog boxes. You just choose a tool that provides sufficient speed or accuracy to meet your needs. But don't expect too much from simple tools. Tools that use image thresholds, for example, cannot provide subpixel accuracy on edge locations.For more complex tasks that fall farther from the origin, you may need one tool to locate an object and a second tool to make dimensional measurements. Because fewer vendors offer tools for these applications, you may need to use image-processing operators such as edge extractors, filters, and morphology tools to transform an image into data suitable for analysis by the four types of tools already discussed. These additional steps may require custom code.
In all cases, though, you should evaluate machine-vision tools only after you have
- identified all possible defects,
- adjusted lighting for the best possible images,
- accounted for positional variations, and
- assembled a test set of images.
Test images should include those that represent typical as well as worst-case conditions, so you can thoroughly try out the analysis software individually or in various combinations. Even though a complex application may require a bit of hand coding, you should concentrate on analyzing the results of your tests and configuring available tools, not on writing machine-vision software. You may find tools that eliminate the need for custom software, and the use of standard tools makes it easier to maintain and upgrade an application.
| Author Information |
| Irene Leszkowicz works as the Group Leader for Interactive Imaging Products at Matrox Imaging, and has over 20 years of experience with the company. She holds a BEng in Electrical Engineering from McGill University and an MASc from the University of Toronto, and is also an Adjunct Professor in Electrical and Computer Engineering at McGill University in Montreal. |
| Sarah Sookman is the Media Relations Specialist at Matrox Imaging and worked as a technical writer for Matrox Imaging before joining the Marketing and Communications department in 2002. She holds a Bachelor of Education degree from McGill University. |
| Reference |
|
