Color enhances inspection results

SIDEBARS: -A “pixel” has two meanings -Monochrome cameras sense colors READ OTHER NOVEMBER ARTICLES:  Contents, November 2006

In the electronics industry, the use of color-vision systems lets test engineers monitor the placement of colored components, differentiate between components and assemblies, and perform other inspections that rely on colors. One application might require a vision system to merely distinguish between a blue connector and a red one, without determining the exact shade of blue or red plastic the connector manufacturer used. In another case, an inspection of keyboard backlights or miniature LCDs may require careful matching of measured colors against standard reference colors.

Adapting a machine-vision system to take advantage of color imaging involves more than simply replacing a monochrome camera with a color camera. You must carefully assess camera characteristics and lighting requirements before jumping on the color-vision bandwagon.

Most color cameras used for inspection tasks fall into two categories: those that use one charge-coupled device (CCD) and those that use three to convert light into electrical signals. In a 1-CCD camera, a transparent color-filter matrix overlays an array of light detectors. Each detector, which corresponds to a pixel in the resulting image, has a minute red, green, or blue “window” above it (Figure 1). A 3-CCD camera includes an individual CCD for each color (red, green, and blue). Because 1-CCD cameras predominate in machine-vision systems, I will cover them exclusively in this article.

Figure 1. This pattern of colors, found in a Bayer-type filter, overlays a CCD in a color camera. Each colored element covers one photosensitive detector, which corresponds to a pixel in a final image.

You may wonder how a camera can create an image of colored pixels if individual detectors respond to only one color of light. Vision equipment relies on math to fill in the “missing” colors for each pixel that will appear in a final image.

“Color-sensor vendors place a color-filter array over the sensor's individual detectors, or pixels. If you have a pixel with a green filter, the incoming light may contain red and blue wavelengths that pixel cannot detect,” explained Steve Kinney, product manager at JAI Pulnix. “So, the camera looks to neighboring pixels to provide the needed information so it can approximate the other two color values for a given pixel.” (See “A 'pixel' has two meanings,” below.)

Figure 2. Software combines red, green, and blue color information from a central detector and the detectors that surround it to interpolate the color value for the resulting image pixel. Although each detector produces an n-bit value for its measured color, the interpolated color information for each pixel comprises 3n bits.

Keep in mind the analog-to-digital converters (ADCs) internal to a 1-CCD color camera produce an 8-bit or 10-bit value that corresponds to the amount of light that reaches each detector. Mathematical interpolation creates a new 24-bit or 30-bit value for each pixel in the resulting image.

A monochrome camera with a 600x400 array of detectors, for example, produces 240,000 bytes per image. A color camera with equivalent resolution produces 720,000 bytes per image. (Both examples assume an 8-bit ADC.) So, when interpolation takes place within a color camera, your host computer must handle an image data rate that is about three times the rate for an equivalent monochrome camera. (CCD configurations of detectors are not always equivalent, so the 3X factor can vary slightly.)

Instead of putting out interpolated color values, some progressive-scan color cameras used for machine-vision applications provide “raw” color data from the individual red, green, and blue detectors. “Do not store that raw information in a JPEG or other compressed format,” warned Kinney of JAI Pulnix. “Compression will ruin the unprocessed color information for an image.”

Software in a host PC will interpolate the pixel values to produce a final color image for processing. Keep in mind the computer time needed to convert raw data into final images. Often, a basic PC cannot keep up with a 60-Mbytes/s flow of raw 8-bit color data, because it also must interpolate that information to create a new 24-bit value for each pixel in a color image. The interpolation creates pixels at a rate of 180 Mbytes/s.

In some applications, you might use only the raw monochrome output from a color camera to inspect the majority of a PCB to, for example, detect the position or orientation of ICs or determine whether components are missing. You could then perform a color-interpolation on only the small areas of an image for which you need color information. Thus, software will look for a red or green connector only in the spot where the connector should exist. (See “Monochrome cameras sense colors.”)

Be aware that you will lose resolution when moving from a monochrome camera to a color camera with an equivalent-sized CCD. Developers often ask camera suppliers to specify how much resolution an image will lose, but the vendors cannot quantify it. If you have a black-and-white edge, you have no color on the black side and all colors on the white side, so regardless of color, detectors are on or off. In that case, you don't see much effect from interpolation in the final image, and the edge appears sharp. (Adjacent high-contrast colors such as blue and red also produce sharp edges.)

The right side of this screen display shows all colors available in a red-green-blue color "space." A point on the orange block and another on the red block translate to the two points shown on the color-space diagram. This type of information assists during system setup and color matching. Courtesy of National Instruments.

Unfortunately, the algorithms may cause a problem: They can remove the defects you want an inspection to reveal. Suppose you have a defect that appears in two or three pixels along a sharp edge. The algorithm “sees” the defect and removes it to “clean up” the edge. You need to pay attention to how software processes images and how that processing can affect results. Often, less-complex algorithms can quickly find defects in an image, even though they do not produce eye-pleasing images.

Unlike monochrome vision systems that simply detect brightness levels across shades of gray, color systems depend on carefully controlled light sources. These sources should maintain a constant intensity and color output, or color temperature.

If you have little or no experience with lighting for a color-inspection system, seek professional advice from lighting vendors or system integrators. Vendors offer a wide range of light sources for a reason: Conditions and requirements vary so much that a small selection of light sources cannot meet all color-inspection needs.

Your choice of light sources will depend on what you want to inspect. “If you need to find a red object and do not need to match an exact shade of red, you may accept some light variations,” said Kyle Voosen, vision product manager at National Instruments. “But if you must measure the color to ensure it matches a production specification, you need lights that produce a known spectrum and maintain it for a long time.”

Halogen lights, for example, produce illumination across a continuous span of wavelengths. Some light sources, though, offer less than meets the eye. Although some LEDs create what looks like white light, their output may lack energy at several wavelengths. And as the phosphors in white LEDs age, the wavelengths of the “white” light shift. Those changes affect the colors a camera captures.

The Panasonic GP-US522HB camera head connects to a stand-alone camera controller that developers can integrate into a color-vision system. Courtesy of Panasonic Medical Vision.

“The intensity of some light sources decreases as they age,” noted Brent Runnels, systems engineer at National Instruments. “Many illumination products now include a feedback system that maintains a stable light output, even as a bulb changes its characteristics over time. Metal-halide lights also can change color over their life.”

How you plan to use a camera will also affect your choice of lights. Monochrome cameras may take advantage of strobe lights that produce high-intensity bursts of light.

“You should approach gas-discharge lights such as fluorescent lamps or strobe lamps with care when you plan to use them with color cameras,” said Richard Erickson, senior product manager at Hitachi Kokusai Electric America. “These lights have characteristics that affect their brightness and color temperature as they turn on or off. If you capture images as a lamp turns on, turns off, and in the middle of a flash, the images will show different colors for the same target.” You could try to synchronize lamp and camera triggering, but some color cameras do not allow for an external trigger. They only operate in a free-run mode.

Generally, a white “target” illuminated in a camera's field of view provides a known reference for color settings. “For a color camera that produces an 8-bit value, you illuminate the white target and adjust the camera so the ADC for each color puts out a full-scale value of 255 for every detector,” explained NI's Runnels. To avoid saturating the detectors, you could “back off” the balance adjustment so each ADC produces a value of less than 255, say 250, for white light. Then, you can set the image-analysis software so it knows the value of 250 for each color represents the contribution from white light.

“If your lights or inspection setup change, you can put a white target in front of your camera and acquire a new image to calibrate the color-balance settings,” noted Runnels. “If you find specular reflections or glare from a product routinely saturate detectors, you could change to diffuse illumination, add a polarizing filter, or take other measures to avoid saturating detectors. Or, you may not care about saturation if it occurs in areas you don't need to inspect.”