What Is Machine Vision All About?

An introduction to machine vision can help you understand the technology and how to apply it.

Jon Titus, Editorial Director -- Test & Measurement World, 7/1/2000

Machine vision—often called automated optical inspection (AOI)—finds application at many points along a production line in the electronics industry. Applications include examination of semiconductor wafers, components, PCBs, complete assemblies, and entire products. By using automated-inspection techniques, manufacturers eliminate the lack of repeatability associated with human inspectors, and they increase both the rate at which products get inspected and the accuracy of the inspections.

You can best start understanding machine vision by learning about five basic types of operations: comparing images, locating a specific component, locating specific features, measuring dimensions accurately, and decoding informational marks. Machine-vision systems perform other operations such as shape identification, parts counting, and so on, but these five provide a good overview. (To keep this article simple, I’ll cover machine-vision as it applies to production of PCBs.)

Comparisons aren’t limited to determining presence or absence, however. They also can use quantitative information to help operators make decisions. A vision system can inspect solder connections and compare characteristics of an acquired image with thresholds that define acceptable limits on quality. Operators or test engineers set these thresholds based on experience with known-good solder connections or other known-good features they need to inspect.

3. Locate specific features or marks. Most machine-vision systems also must locate specific marks, often called fiducial marks. Depending on the product undergoing inspection, these “landmarks” could indicate where a machine-vision system must start acquiring images (some large boards require several images), or they may help equipment properly align a PCB for the next assembly step. The fiducial marks also let a machine-vision system reorient an image to account for position changes during production. Not all assemblies get held rigidly as they pass an inspection station. Instead, they get placed randomly on a conveyor belt by production equipment. Thus, the software must account for position changes. Software may include pattern-matching routines that locate a complex pattern regardless of its size, orientation, or location.

Figure 1. Two images of pin headers show the measurements a machine-vision system can make based on acquired image information. An operator had to calibrate the system by providing a factor that lets the software convert numbers of pixels to a mechanical length. (Courtesy of National Instruments.)

Figure 2. A three-dimensional imaging system scans across an image to acquire x- , y-, and z-axis information about an object. A computer can then manipulate the information and produce images for analysis. (Courtesy of Intelligent Automation.)

Figure 3. Information encoded in a Data Matrix symbol appears in a grid of black-or-white squares. The grid includes alignment and error-correction information. The grid comes in different sizes, depending on the amount of information encoded in the pattern. (Courtesy of RVSI Acuity CiMatrix.)

4. Measure distances and angles. If a manufacturing system places a multipin header on a PCB, it also should ensure the pins are straight, parallel, and properly spaced. A machine-vision system can examine the connectors after assembly and measure both the distances between adjacent pins and the angle between the pins and a reference, probably the edge of the PCB (Fig. 1). The machine-vision system would “fail” any PCBs containing bent or misaligned pins. These types of measurements fall under the general headings of gauging and metrology.

Gauging comes into play when the system must measure the size of components or the position of components relative to nearby fiducial marks. A vision system would use gauging operations to ensure a hole in a component and its mounting hole in the PCB align within spec prior to soldering. You might want the holes properly aligned for later attachment of a heat sink.

Most inspection systems are inherently two-dimensional, so they cannot measure depth. For example, commercial systems that inspect solder-paste deposits on PCBs use 2-D information to measure the size and shape of the deposits. Production people who understand the nuances of a specific paste-printing process can make inferences about the process and the volume of solder from the 2-D information alone.

Some vision systems, though, require depth information for their analyses (Fig. 2). These 3-D systems rely on techniques that use moiré patterns or lasers to collect depth information that lets them measure solder-paste volumes. Accurate volumes help improve the quality in a process that places small volumes of solder paste on fine-pitch PCB contacts. But unless you specifically need 3-D information, 2-D systems will work well.

After a machine-vision system acquires an image, how does it use the information? At its simplest, the vision system would indicate a pass or fail condition to an operator, and the operator would remove defective products from the production line. It’s more likely that the vision system works with other production-line equipment, perhaps automatically sorting products into pass or fail bins.

A more complex vision system could identify a failing assembly and route it to a rework station. Information such as “Assembly 456GH: Missing R57 chip resistor” would appear when a repair technician scans the assembly’s bar-code label. And if the vision system provides a network connection, test engineers can easily distribute and save quality-assurance reports and data.

Moving Along the Line
Unlike more expensive electronic testers, machine-vision systems find use at many points along a production line. The following example shows how a hypothetical manufacturer of PDAs (personal digital assistants) might use vision systems on a production line. (A real manufacturer might use more or fewer inspection stations, and actual production would involve more steps.) My example assumes an individual board moves along a production line. In reality, panels, or frames, of several mechanically connected boards moves along a line as a unit.

At the start of production, a machine-vision system inspects a bare PCB and reads its label. This inspection ensures that operators feed the proper PCBs onto the production line and that the PCBs are properly positioned. At this point, the vision system also might record a revision number etched on the PCB during fabrication. The revision number will find use later in production.

The board then moves down the production line to the solder-paste printing machine that applies solder paste to contacts. After application of the paste, the PCB undergoes an inspection that ensures the contacts have received the proper amount of paste. If a fault occurs, the system notifies the operator to remove the failed PCB from the production line.

Quantitative information about solder-paste defects can tell the operator that the printer is slowly drifting out of spec or that the paste itself has problems. The inspection information also may form part of a statistical process control (SPC) database that operators use to chart and analyze quality information. Operators can use the SPC results to take remedial actions before too many boards get rejected.

At the next production stage, pick-and-place equipment populates the PCB by rapidly placing components in designated positions. The pick-and-place equipment may come with an integrated vision system that measures the characteristics of fine-pitch leads on surface-mount ICs. This inspection step ensures the ICs will properly contact the solder paste on the PCB. Internal metrology software measures distances between contacts and the coplanarity of contacts. By making these visual measurements, the system can reject out-of-spec ICs before they get placed on a PCB—and show up as rejects later down the production line.

Figure 4. Vision systems often require special packaging for industrial applications. The camera resides inside the black enclosure that protects it from rough handling.. Multicolor LEDs mounted at high and low angles inside the housing illuminate PCBs passing the inspection station on an assembly line. (Courtesy of ViTechnology.)

After the pick-and-place equipment finishes its tasks, the PCB undergoes inspection again to ensure components are where they’re supposed to be ( Fig. 4). Depending on the manufacturer’s needs, the vision system at this point could inspect for specific components at specific places. OCR software would compare component markings with expected markings. Defect information may reveal problems with pick-and-place equipment or with the components loaded into the equipment.

Next, the PCB moves through a reflow oven that melts the solder. The surface tension of the molten solder usually aligns component leads and PCB pads. But once in a while, small components such as chip resistors and capacitors “tombstone,” or sit upright on the PCB. A post-reflow inspection locates any components that look unusual or that have moved outside preset tolerance settings. The vision system also may examine the quality of the solder connections by looking at the geometry of the reflowed solder and at the solder’s
reflectance.

You may wonder how a vision system can check the quality of solder connections hidden under packages such as BGAs. The inspections in this example have used visible light, but not all inspections do so. Checking BGA solder connections requires x-rays that pass through a BGA package or other materials to reveal the structure of solder connections. Software compares the acquired image information with preset tolerance limits.

Toward the end of the hypothetical PDA production line, an LCD gets mated to each unit’s main PCB. The display then needs testing, which may occur coincident with functional testing. As the ATE runs through electrical tests, it triggers the vision system to examine the LCD to determine whether the display appears as it should. The inspection results feed back into the electronic tester to indicate success or failure. A failure may abort further testing or call for diagnostic tests, either online or off. Assuming the unit passes all its tests, the revision number, scanned at the start of production, indicates to the functional tester what software to load into the product’s microprocessor.

After final assembly of the PDA, another inspection station checks the product for appearance, proper appearance of button legends, and other characteristics. Although not specifically described above, each of the inspection stations read the PCB’s bar-code label to track the unit through production.

The last vision system on the production line sends quality-control and serial-number information to the company’s corporate database. If a unit comes back with a field failure, quality-control engineers can determine when the company manufactured the product, what component lots it used, what version of the software it used, and so on.

Applications and Costs Vary
The use of machine-vision systems varies from manufacturer to manufacturer, depending on the manufacturing processes they use and the products they build. Companies producing large quantities of simple products may rely on simple machine-vision systems to check component placement or to perform a final visual check. (If the product is simple enough, they may choose to use human inspectors.) On the other hand, a manufacturer of complex workstation motherboards might use a machine-vision system at many production steps. This manufacturer cannot afford to extensively rework costly motherboards or to process field failures. It must catch production errors and correct them immediately.

As you evaluate the potential of machine-vision systems, you must weigh the tradeoffs of costs for the systems (including costs for operators and training) against gains from improving quality and reducing rejected products. Include in your balance any changes machine-vision systems will make to the way your company produces products. You may have to consider slower production lines, different tooling, and additional equipment on the production floor. Still, machine-vision systems can add to the quality of a product and reduce other test-related costs.

How Does a System Come Together?
All vision systems—whether they come as part of a handler or ATE, or as part of a home-grown PC-based system—use similar components as described below. (I won’t describe x-ray systems components, because you buy entire systems.)

Figure 5. Lights used in machine-vision applications come in many shapes and colors that engineers choose to suit specific needs. Some applications may require several types of lights, controlled by a computer, to properly light different assemblies or components. (Courtesy of CCS America.)

Figure 6. Special fiber-optic heads use fibers to place light right where it’s needed. In this example, the manufacturer has arranged the individual optical fibers to provide a bright line of illumination for a line-scan camera. (Courtesy of Stocker & Yale.)

Lighting
It may surprise you, but lighting forms the most critical piece of a machine-vision system. Without proper lighting, the rest of the system can’t capture an image worth processing. Unfortunately, people who engage in do-it-yourself machine-vision projects usually leave lighting considerations for last. Lighting specifications include the type of lighting—monochrome, laser, white light, flash lamps, and so on—and also the configuration of the light sources—line sources, ring source, area source, and so on. Figure 5 shows many types of LED light sources used in machine-vision systems. Light sources come in many colors and shapes, but factors such as the angle of illumination, brightness, and coverage of the subject also figure into system specifications.

Failure to obtain the proper lighting can doom a machine-vision project. Even the best software and the best camera cannot compensate for poor lighting. Fortunately, several suppliers specialize in lighting products for machine-vision applications, and they provide applications assistance and information.

Cameras
Electronic cameras capture a visual image and turn it into electronic signals a computer can process. Most general-purpose machine-vision cameras operate with an 8-bit gray-scale resolution. Thus, they divide the light range from black to white in 2⁸, or 256, gray-scale steps. Camera manufacturers sell units with 10-bit resolution; higher resolutions are available for research and lab work.

Cameras fall into two general categories: area-scan cameras and linescan cameras. Area-scan cameras use a 2-D array of detectors to capture an entire image, much like a snapshot camera used for still photography. A linescan camera uses a single line of sensors to “scan” an image, a line at a time.  Linescan cameras prove useful when you need to capture an image of a moving product, a long object, or perhaps a cylindrical object. The camera acquires an image a line at a time as an object moves by it. And a linescan camera requires only a line source of light (Fig. 6). (Actually, a PC builds the image as the camera acquires each image line.) If you rotate a cylindrical object in front of a line camera, the camera “unwraps” the surface into a single long image.

Area cameras split into two types—those using interlaced scan and those providing progressive scan. The former works like a TV camera. It acquires an image and scans first the odd-numbered lines; then it acquires another image from which it scans the even-numbered lines. A computer connected to the camera meshes the lines to reproduce a complete image. But because the camera requires two sequential images, a fast-moving object can produce a slightly blurred image. Interlaced-scan cameras work well if an object remains motionless while a camera acquires its image. These relatively inexpensive cameras serve well in many machine-vision applications.

A progressive-scan camera captures an image in one shot and scans out the lines one after the other. Its images aren’t subject to as much blurring as those from an interlaced camera. Many progressive-scan cameras come with sophisticated shutter and control options that let a PC carefully adjust image acquisition to meet specific inspection needs. The level of control provided by progressive-scan cameras makes them a good choice for vision systems.

Most applications in the electronic industry use monochrome (black-and-white) cameras. Camera manufacturers do offer color cameras, but their cost doesn’t justify their use unless an application demands color. Testing color displays, testing the color of LEDs, or matching component colors would require a color-camera. Color cameras produce three times as much information as monochrome cameras because images contain red, blue, and green intensity data. Processing color images takes longer than processing monochrome images, thus affecting throughput.

Except for using a few standard formats such as RS-170, CCIR, and IEEE 1394, camera manufacturers have gone their own way when it comes to providing video information to a PC. So, not all cameras work with all frame-grabber boards, the interface between a camera and a PC.

Frame Grabbers
A frame-grabber board plugs into a PC—usually into the PCI bus—and connects it to a camera. Thankfully, frame grabbers come with a list of compatible camera types so tracking down a match doesn’t take long. The frame grabber accepts the camera’s video-output signals and reassembles the video information into a matrix of values. The values represent the intensities for the corresponding pixels in the image.

After producing the matrix of values, the frame grabber usually transfers the information into the host PC’s memory for processing. Some frame grabbers transfer the video information into the PC’s memory as it arrives from the camera. More advanced frame grabbers can perform signal-processing operations that enhance an image, remove noise, and so on. A few sophisticated frame-grabber boards include an onboard processor that lets them operate on image information with minimal interaction from the host PC.

In addition to accepting signals from a camera—or sometimes from several cameras—frame grabbers may supply signals that let application software control a camera’s shutter speed, integration time, and image-acquisition triggering. Some frame grabbers include separate digital I/O lines that application engineers can use to detect sensor conditions and control production-line equipment.

Computer
Because popular frame grabbers work in a host PC, many machine-vision systems use standard PCs that deliver high-speed operation, plenty of memory, and easy network communications. Unfortunately, today’s PCs offer few PCI expansion slots, so vision-system developers must choose configurations of frame grabbers and PCs carefully to get as much as they can in one package. Cameras that use the Firewire interface standard (IEEE 1394) can get away without frame-grabber boards, but today only a few cameras provide this type of output and only a few PCs support 1394 connections.

The CompactPCI bus and the PCI Extension for Instrumentation bus, or PXI bus, provide some relief. These buses offer the capability to expand PCI-type systems beyond the three PCI slots in a PC. A PXI chassis, for example, may include eight slots. Several manufacturers offer CompactPCI and PXI frame-grabber cards, and manufacturers still supply frame-grabber and image-processing boards for VMEbus systems.

Manufacturers also supply proprietary vision systems that include their own hardware and software. In most cases, these companies sell their equipment to system integrators for resale to end users. Some machine-vision-system suppliers offer specialized computers as well as stand-alone machine-vision subsystems that perform many inspection tasks without relying on a host PC (Fig. 7). Nonetheless, these small systems can easily communicate with PCs to pass along inspection results and quality information.

Software
After acquiring an image, a PC must process it to yield useful information to equipment operators. Software comes in a variety of formats, from stand-alone, rapid-development packages to libraries of C++ routines. The rapid-development packages let you quickly “connect” functions, in either graphical or text format, to construct an application. These packages come with most of the functions a developer needs to get started on a machine-vision application. T&MW