Basic Functions Ease Entry Into Machine Vision

Another function, the histogram, uses all the pixels in an image, but it simply counts the number of pixels that exist at each of the 256 possible gray levels in an 8-bit image. The histogram function plots the number of occurrences vs. pixel intensity (gray level). This simple plot shows whether an image presents enough contrast between objects for software to differentiate them.

The image in Figure 2 shows an 8-bit image of metallic iron as it appears in iron ore. The lighter objects in the image represent the metallic iron. The image’s histogram (Fig. 3) displays the intensity distribution for the pixels. The background appears at gray level 40, the ore shows up at gray level 70, and the high-intensity iron turns up at gray levels above 150. From the histogram information, you can determine that the image clearly differentiates the components. Thus, an image-analysis system can “separate” the iron from the ore just by identifying the corresponding gray levels in the image.

Functions Analyze Shapes

The histogram information helps software accurately and effectively filter features and objects for subsequent morphology and blob analysis. For the beginning machine-vision developer, these terms—morphology and blob—sound odd or humorous. Machine-
vision experts, however, use these functions often to locate and analyze shapes. In the electronics industry, you can use morphology and blob analysis to locate, count, and size defects on PCBs and on semiconductor wafers. You can even use them to analyze the spray patterns of fuel injectors.

A blob isn’t a pile of menacing slime. It’s an abbreviation that stands for binary large objects. The image undergoing blob analysis exists only as zeros or ones—binary—information. Thus, the first step in blob analysis involves setting a threshold for the image to divide it into zeros and ones. If you set the threshold to gray level 148, for example, all the pixels with gray levels of 148 or lower get set to zero. Pixels with gray levels of 149 or higher get set to one as shown in Figure 4.

The threshold value doesn’t get chosen at random. You must use the histogram of an image to guide your threshold selection as shown in Figure 3. Blob analysis software can now locate contiguous groups of logic-one pixels to count individual objects. Overall, blob analysis aids software development and image processing by simplifying images.

You also can use blob analysis to compute the area of objects in an image. But the software does not know a pixel represents an area of x mm² in an image, so before you can get the software to compute areas in standard units, you must manually provide the area conversion factor: 1 pixel = x mm². To compute an area, the software counts the number of logic-one pixels adjacent to other logic-one pixels and keeps a running sum. The sum of the pixels corresponds to the area of the object in square pixels. Then, the software multiplies the pixel counts and the pixel-to-mm² conversion factor to get the area in square millimeters. Software can also calculate the perimeter of an area by counting the number of pixels that have at least one logic-zero pixel adjacent to them. Thus, the software counts only pixels along an outer edge.

You can use blob-analysis functions to calculate more than 50 parameters such as area, perimeter, center of mass, longest segment, bounding rectangle, circularity and so on. For the iron ore application, the blob analysis function returns the center of mass (x-y coordinate) and the size of each object.

Morphology Manipulates Shapes

Morphology functions complement blob-analysis functions. In general, morphology functions slightly change the shapes of objects so software can easily count them. For example, an erosion morphology function reduces the perimeter of objects to separate objects that are close together or partially overlapping. The erosion also eliminates small image features such as ragged edges and individual “noisy” pixels. You can think of morphology functions as acting like high-pass and low-pass filters.

In the same way, the erosion function can also fill in small “holes” inside an object. These holes may result from noise, or from reflections off dust or small imperfections. Using the image in Figure 4 and performing an erosion function yields the image shown in Figure 5. Morphology functions fill holes, remove small blobs, and remove border objects in order to remove unwanted pixels and simplify the counting and sizing of objects.

Morphology and blob-analysis functions provide powerful inspection tools, but they have limitations. They work best when analyzing images that provide good contrast between objects and the background. And the imaging system must provide uniform lighting so no lighting gradients appear in the image. That means the same types of objects must appear at the same, or very close, gray levels in every portion of an image.

Derivatives Locate Edges

In addition to computing areas and perimeters, a machine-vision system may have to measure, or gauge, actual dimensions in an image, usually between edges of an object. Gauging finds use in inspection of mechanical assemblies, electronic packages, and electronic connectors.

Figure 6. A properly assembled connector shows individual pins properly spaced and straight in a molded plastic housing.

Figure 7. The measurement of pin-to-pin distances and angles along each pin let software detect a damaged connector.

Typically, software “locates” edges using the pixel values in a straight line across the area in which you expect to find the edge. The software examines the pixel values to locate rapid changes in gray levels. You probably recall from calculus that the derivative of a continuous algebraic function locates any inflection points. The inflection points represent the points at which the slope changes from positive to negative, or vice versa. Many edge-detection algorithms take the second derivative of the pixel values to locate edges along a line. An edge-detection function can use intensity information to interpolate between pixels and resolve an edge location to sub-pixel accuracy. Keep in mind that the edge-detection functions operate only on the pixels in a line, not on an entire image. When they detect edges, they convert the information to coordinates that define specific locations.

An example shows how edge detection applies to a real inspection task. The image in Figure 6 shows portions of a connector with straight pins, and Figure 7 shows a similar connector but with two bent pins. The vision system has been programmed to calculate the distance in pixels between each pin at three points along its length. The system also uses the three points to determine whether or not each pin is straight, as determined by the angle between the three points. The three points should exist along a straight line, or an angle of 180°, within a preset tolerance. If the distances and the angle do not fall within preset the tolerances, the system rejects the connector as flawed.

Edge-detection techniques also have other applications. To determine whether or not a part is present on a PCB, software can acquire a line of pixels across the component and count the number of edges. It then compares the count to a preset value and when the values match, the component exists. Unlike blob-analysis and morphology functions, you can use edge-detection functions when images contain lighting gradients. The edge-detection functions simply respond to rapid, local changes in intensity. Small, gradual lighting gradients do not affect these functions.

Match a Pattern, Too

Often, vision software must match a pattern in its memory to a pattern on a PCB or semiconductor wafer. Pattern matching comes into play when an application must locate specific targets, or fiducial marks, on a PCB or assembly. By locating several fiducial marks on a PCB, for example, a machine-vision system built into a parts handler can properly align a PCB, inspect components relative to their locations on the PCB, and guide component placement.

In pattern-matching operations, the known pattern—often called a template or model—and the image possess gray-scale information. Thus, software can mathematically correlate the images and test for the best good match. The process of matching two patterns approximates that of correlating two measurement signals. Pattern-matching functions must take into account actual production-line conditions such as unpredictable changes in illuminations and other adverse process variations such as a partially hidden pattern, a rotated pattern, a scaled pattern, and a camera slightly out of focus. And any pattern-matching software must work fast enough to keep up with its associated production line.

The images in Figure 8 show the template or model image of a fiducial mark on a printed circuit board (PCB) and how pattern-matching functions locate the fiducial mark under various conditions. Pattern-matching software can locate complex patterns in a full image (640x480) in less than 80 ms, and the objects can be at any orientation. For fast location, you can limit the search to find objects that are not rotated. In this mode, you can find objects as fast as 10 ms with a 100x100 template image.

a)	b)
c)	d)
Figure 8. Robust pattern-matching software locates a fiducial on a PCB even under varying conditions such as (a) a blurred image, (b) a partially occluded image, (c) a lighting change, and (d) a rotated image.

To use pattern matching, you first define a “golden master” or “template” that represents the object you want the software to find. The software “learns” the template by collecting shape, background, and edge information from it. If the software must find a rotated object, the learning operation can take a few seconds depending on the complexity of the template. Then, the software correlates the template information to an acquired image. Typically, the search of an image for a specific feature starts in the upper left corner and scans across the image ending at the lower right corner.

Pattern-matching algorithms first often use a fast, but less accurate, technique to locate likely candidates for later application of a high-accuracy correlation algorithm. That algorithm supplies the x-y coordinates of the identified pattern. The image-processing steps typically take a few milliseconds for a 640x480 image. The coordinates are accurate to subpixel dimensions because the pattern-matching software interpolates the center of the pattern between surrounding pixels. The software also returns the angle of rotation, if any, to an accuracy of 0.1°. A motion-control system can use the x-y coordinates and the angle measurement to properly align and guide a product at a component-placement station, for example.

The best way to learn more about the types of operations described above is to buy some image-processing software—or get a demo package from a supplier. Then, you can try your hand at applying functions to images to see how the processing affects the image’s information. T&MW

FOR FURTHER READING

Baxes, Gregory A., Digital Image Processing: Principles and Applications, John Wiley & Sons, New York, NY. 1994.

Jain, Ramesh, Rangacher Kasturi, and Brian G. Schunck, Machine Vision, McGraw-Hill Higher Education, Burr Ridge, IL. 1995.

Masi, C, G., “Lighting Makes Its Mark on Vision Systems,” Test & Measurement World. May 1998, pp. 11–14.

“PC-Based Vision Solutions” National Instruments, Austin, TX.

Sonka, Milan, Vaclav Hlavac, and Roger Boyle, Image Processing, Analysis, and Machine Vision, 2nd ed., Brooks/Cole/PWS, Pacific Grove, CA. 1999.

VanDommelen, Carl H., “Choose the Right Lighting for Inspection,” Test & Measurement World. October 1996, pp. 53–58.

(The articles from T&MW are available online at www.tmworld.com/archives.)

John Hanks is the vision and motion-control strategic marketing manager at National Instruments. In his 10 years with the company, he has worked in strategic marketing, product management, sales management, and applications engineering positions. He has a B.S. from Texas A&M University and an engineering M.S. degree from the University of Texas at Austin.