Another point of view in machine vision

State-of-the-art image-processing techniques place a huge strain on computer resources: The number of operations necessary to completely analyze an image increases as the square of the resolution increases, so every doubling of image resolution quadruples the required computer resources. Rising computer speeds and memory capacities are unlikely to keep pace with these growing demands.

Machine-vision systems that rely on brute-force pixel processing will soon be inadequate for many applications. The processing of complete images has become so complex that vision systems often can examine only small segments of the target area to keep inspection time within allowable limits, attempting to draw any necessary conclusions from those segments. The process of identifying and classifying an image (answering the question, "What am I looking at?") has become the most critical part of the operation—and the most error-prone. Even looking at the "right" image subset doesn't always guarantee success, because normal image-to-image variations that humans consider trivial will confound the most sophisticated conventional computer tools.

A promising alternative to brute-force computational techniques exploits technology and machine resources differently. It employs methods that humans use to process visual information.

When a computer uses brute-force analysis to compare images, it divides one image into subsets, examines the subsets, and compares them with corresponding alternatives in a reference image, looking for a precise (within the limits of error) correspondence. With simple objects that offer little sample-to-sample variation, such correlation methods work well enough. With wider variation, however, they fall apart.

Consider a vision system that compares a stable object—say a fingerprint or a retinal pattern—to a stored idealized representation. The object and the reference are essentially identical, so the computer can compare them quickly and efficiently despite the hundreds or thousands of individual elements that it must include to establish a match.

But the comparison of two otherwise similar objects that differ in ways not necessarily relevant to the end-purpose—size, color, or markings on a semiconductor, for example—creates a problem for all correlation-based methods (including brute force). Such methods weigh all differences equally, whether they are relevant or not. A more efficient technique would perform human-like judgments to decide which differences to ignore and what level of correlation is sufficient.

It is even more difficult for a machine to recognize a collection of objects. In two variations of a circuit board that contains a collection of components, the color of each component could vary to some degree, and the board itself could be rotated 90°. The task of recognizing the board is separate from the task of examining the individual components.

Paradoxically, increasing image resolution can aggravate the situation. In low-resolution images, the pixels are larger and there are fewer of them, masking minor differences between the test image and the master. Higher resolution means more pixels per unit area, which multiplies the number of comparisons required, increasing the likelihood that the machine will perceive the boards as different. Determining an optimal resolution that reduces analysis complexity and still accurately recognizes a collection of variable objects represents the fundamental challenge for developers of vision systems for PCB inspection.

Conventional machine-vision-based inspection systems generally depend on specific light-source characteristics (direction, color, intensity, angle, and pattern) and multiple cameras to provide optimal illumination and allow analysis of critical elements of the unit under test (UUT). The user must adjust the inspection conditions to achieve consistent pass-fail performance. So, image analysis, positioning of the light-source and camera, and establishing pass-fail boundary conditions all become interactive rather than fully automated activities. The machine can only report that one region measures 90 counts (in whatever unit of measure) higher than another. A human must decide: Does that figure mean the machine has found an edge? Is the area sufficiently lighted or colored? Is there a discernable transition? Do these conditions together mean that the target object has passed the test?

Although computer systems transfer and manipulate objects and scenes effectively, the techniques for recognizing those objects and scenes lag far behind. Using a browser and a search engine, you can type in a keyword and retrieve related images from Web sites and graphics software, but only if someone has assigned keyword categories to those images and attached appropriate text descriptors. No reliable tool exists that can analyze the images themselves and identify them automatically.

As a result, programming so-called "automatic" optical-inspection systems remains a largely manual pro-cess. Current vision-based PCB inspection techniques succeed best in high-volume/low-mix situations that require a minimum of program changes.

To broaden the appeal of inspection techniques, we propose a system based on biological rather than pixel-based principles. This approach can overcome limitations imposed by variations in light sources and the objects themselves. Manufacturers could use ordinary lighting and camera conditions when collecting images and then evaluate the images with sophisticated software that would recognize normal variations and compensate for them the way humans do.

We humans know a priori what we're looking for. We internalize features like shape, color, and size as collectively describing an object. We also expect to find these features in a familiar context. Object and context information help categorize what we are looking at, simplifying more explicit recognition of the object in the field of view.

We don't always recognize an object when it is not in the expected context. When you park a new car in a shopping center near a light pole, you use the context of that pole to help locate the car later. If your spouse moves the car before you return, you're much less likely to recognize it—even if you walk right by it—because the context has changed. If you do find it, you confirm your identification by examining the color, the license plate number, and the hubcaps—top-down descriptors of the object that you expected to find.

In addition to context, we use holistic recognition to classify scenes even without fine detail. Blurred images created by filtering out high-frequency content (or expressing the image in coarse pixelation) are still reliably recognizable, as in the image in Figure 1.

A vision system embodying holistic recognition for PCB inspection would include information about the appearance of components, solder joints, paste bricks, board pads, copper traces, PCB materials, and other objects as well as information about the context in which those objects appear. This stored version must accurately represent what the system is looking for—a PCB in this case—without distortion and without assuming in advance absolute PCB color, luminance, markings, and so on. The machine must know enough about the process to distinguish between relevant and irrelevant variations.

Holistic recognition includes visual cues that come from the context in which they are found. The context of the two sample components in Figure 2 comes from the background on which the objects are placed. Visual cues in that background—the dimensions of the occlusion of the pad-and-paste regions, and the differing appearance of the background itself—provide visual cues on whether the object is missing. A recognition system incorporating these visual cues delivers higher detection rates than one that does not, because the system uses multiple conditions (context and object) to make the decision.

In spite of the fact that color is generally considered a normal variation, in some cases (Figure 3), it provides part of the context and therefore contributes to the decision. An inspection system could ignore absolute colors, considering only relative colors from one region to the next. Again, the goal is to make the pass/fail decision at as low a resolution as possible, maximizing inspection effectiveness without slowing throughput.

These alternative forms of machine vision promise to advance the state of the art to reduce the spiraling increase in time and computer power required to process images and to eliminate manual programming and tweaking in inspection-system applications.