Home>Design & Prototyping Test Center>Test How-To

Vision software takes two paths

Jon Titus, Contributing Technical Editor- December 22, 2006

READ OTHER DEC./JAN. ARTICLES:  Contents, Dec. 2006/Jan. 2007

Cameras, lights, and computers may be the mainstays of machine-vision systems, but it’s the software that does the work. As a result, software development plays a key role in the creation of vision or inspection systems. Most engineers tackle the development of vision software in one of two ways.

The two approaches differ only in how far engineers remove themselves from software details and how much control over the software they want. In both cases, engineers rely on vendors to provide algorithms that match patterns, find edges, measure image features, and perform other key functions.

In the first approach, an engineer works with high-level programs that provide a graphical user interface (GUI) for connecting equipment-control and image-processing icons to create a sequence of inspection steps. Software vendors refer to programs that offer this feature as point-and-click, drag-and-drop, or configuration-based tools. These tools appeal to engineers who must quickly prototype a system to test its feasibility or who must solve basic inspection tasks that don’t stray far from common requirements. Engineers who create one-off inspection systems often use this approach, too, because it takes little effort to create, maintain, and update an application.

In the second approach, an engineer buys or licenses libraries of image-acquisition and image-processing tools to integrate with code that he or she has written in C, C++, Visual Basic, or another language. This approach makes sense for engineers who plan to build many inspection stations and who must minimize costs. Some engineers use libraries when they need only a few functions, when they must optimize an application, or when they require advanced capabilities in their code.

To provide an overview of the state of vision software for both types of users, I talked with people at five vendors about their products. The descriptions below highlight new or improved capabilities and important features.

Connect the dots

The Vision Foundry from Data Translation comes in three sections: a GUI that contains drag-and-drop scripting tools, an application program interface (API) that gives engineers direct access to the software for all the Vision Foundry tools, and an expandable “container” that lets programmers create their own tools.

The GUI lets users link tools to create a sequential script of vision tasks. A simple script could acquire an image, look for “blobs,” count the blobs, and make a decision based on the number of blobs present or based on other blob characteristics. In the vision industry, where blobs are simply objects in an image, a blob might represent an IC, a socket, a connector, or some other component. Thorsten Beierle, technical support and applications manager at Data Translation, said that within the Vision Foundry package, the blob tool gets the most use, either for using its results directly or as a basis for other tools.

Figure 1. The main display for Vision Foundry shows a sample image, icons that represent operations and vision tools, and a point-and-click script window. The script controls let users step through and edit the script. Courtesy of Data Translation.

Users can also “train” the pattern-matching tool in Vision Foundry to recognize a specific shape. The package will then search a gray-scale image for geometric features and determine when a region of interest within the image contains that shape. The tool determines and reports the shape’s position and its scope—a measure of how well the shape matches a corresponding “golden image” template.

If an application requires more capabilities than a GUI-based script can provide, the system developer can access the same Vision Foundry tools through an API. As a result, when the developer writes code in Visual Basic or C++, he or she knows the underlying algorithms will work exactly as they did in the GUI script. Programmers also can use ActiveX controls that send and receive variables and images.

Routines created with Data Translation’s Measure Foundry software, which controls data-acquisition and I/O boards, link easily with Vision Foundry scripts. Thus, you can easily include electrical measurements and the control of electrical devices within a machine-vision application. If you need to control a robot, for example, you can create your own tool and icon. Then, you can use the tool within Vision Foundry or use it in a C++ or Visual Basic program.

Make a conditional change

Typically, applications based on drag-and-drop vision software execute a “loop” of sequential steps that runs indefinitely. To improve the versatility of this type of software, release 3.0 of Vision Builder for Automated Inspection from National Instruments allows for conditional branching. So, the result of one operation can determine which operation comes next. In effect, vision software appears to operate as a state machine.

Figure 2. This state-machine diagram from Vision Builder AI 3.0 shows how a state machine uses six states to simultaneously inspect pulleys and gears. Courtesy of National Instruments.

“State machines describe applications as a series of discrete steps that move engineers away from implementation details,” said Kyle Voosen, vision product manager at National Instruments. “A state machine only advances to the next state when it detects a specific condition.”

Although the state-machine approach adds some complexity to code development, it comes nowhere near the complexity involved with programming vision applications in programming languages such as Visual Basic or C/C++. By casting a vision application in terms of a state machine, users can add conditional loops and branches common to a language such as C or Basic. But because a single state can include many unconditional steps, even a complex task may need only a few states.

Vision Builder AI 3.0 continues to let engineers use the sequential nonbranching operations offered in earlier versions of the program. The new software doesn’t force engineers to use the state-machine capabilities—it simply treats an application without conditional branching as a single-state state machine. And code developed with earlier versions of Vision Builder will continue to run under version 3.0, as will all National Instruments hardware.

Engineers also can benefit from the run-time versions of the Vision Builder software. They no longer must buy a complete software package for each vision system. Instead, they can develop software on one computer and deploy it on others that have an associated run-time license.

Go to the library

When engineers need a special image-processing capability, or when they want to optimize operations, they must turn to libraries of functions that operate with languages such as C/C++, Visual Basic, and others. Dalsa bases the image-acquisition and image-processing code in its Sapera Essential package on a proprietary technique called Trigger-to-Image reliability. The technique provides a framework that guarantees the unambiguous identification of images from the moment a system captures them until the vision software makes a decision based on image information.

Figure 3. Edge-based search parameters set within the Sapera Essential tools let programmers "tune" the underlying algorithm for rotation, scale, and other factors. Courtesy of Dalsa.

“The Sapera Essential 'infrastructure’ that identifies, tracks, and reports results for each image processed frees users to refine and enhance their core algorithms,” explained Inder Kohli, product manager at Dalsa. “In addition, performance-analysis and prototyping tools let users identify and remedy potential performance bottlenecks.”

To increase the value of the Sapera Essential libraries, Dalsa now provides access to all the image-processing primitives, or fundamental routines. These primitives cover basic morphology, segmentation, and similar operations. Modules build upon primitives and perform higher-level tasks. A simple edge-based search module, for example, can realign an image with a golden template. Rather than licensing an expensive and sophisticated suite of geometric pattern-matching algorithms that does more than you need, you can use small modules that better fit the tasks at hand.

Sapera Essential lets users choose the modules they want to license, and the majority of the modules cost nothing extra. Some modules, such as those for blob analysis and searching, require payment of run-time fees.

Soon Dalsa will offer a prototyping tool for Sapera Essential so engineers can quickly “connect” and evaluate image-capture and image-processing functions. The tool provides algorithms, modules, and other functions that let engineers try “proof-of-concept” code, but it will not create a complete application.

Quantify image measurements

Figure 4. These two images show the areas used to measure characteristics of plastic hooks. a) A good hook passes inspection, while b) a defective hook fails, based on criteria set in software based on the Matrox Imaging Library. Courtesy of Matrox Imaging.

Version 8.0 of the Matrox Imaging Library (MIL) from Matrox Imaging offers engineers an expanding set of tools they can use from within C/C++ programs. Active MIL, a subset of the more expansive MIL package, provides ActiveX controls. Recently, Matrox introduced a metrology module as part of Processing Pack 3, the latest update for the MIL package. The metrology module lets engineers measure characteristics of image features such as arcs, circles, line segments, and points. Because users can calibrate measurements, the module will convert image dimensions to real-world measurements and coordinate systems.

Pierantonio Boriero, product line manager at Matrox, stressed that rather than relying on low-level libraries of code from a microprocessor manufacturer such as Intel, Matrox programmers optimize the MIL code from the ground up. “Our people can optimize high-level algorithms in ways Intel doesn’t address. We go through MIL’s code line by line, and we also write code for Streaming SIMD Extensions 2 [SSE2] operations specific to several Intel and AMD processors.” Those operations handle multiple data streams and vectors, which makes them particularly applicable to image-processing tasks.

Boriero also noted many engineers forget the importance of image displays in vision systems. The MIL package includes functions that display images across multiple monitors, produce nontearing displays, and create nondestructive overlays. Boriero said that the optimized display functions don’t take CPU horsepower away from image-processing tasks.

Buy the tools you need

Although Stemmer Imaging (Puchheim, Germany) has offered its comprehensive Common Vision Blox software for several years, the company now places many image-capture, blob-analysis, edge-detection, correlation-search, and filtering tools in its basic Foundation Package. This package also includes threshold operations, color-space conversions, overlay generators, and calibration tools.

Figure 5. A tutorial for Common Vision Blox lets engineers try algorithms, in this case to determine their suitability for reading bar codes. Users can check off the types of codes they want to read during a test. Courtesy of Image Labs International.

As engineers need additional tools, perhaps for bar-code reading and character recognition, they buy or license them a la carte from Stemmer. “You add only the tools you need,” said Brian Smithgall, president of Image Labs International, the US distributor of Stemmer software. “A free demo CD-ROM includes all the software tools, so engineers can try tools before they buy them,” said Smithgall. “Some people will need only a few tools rather than the full Foundation Package.”

Although engineers can use tools in the Foundation Package with C/C++, the tools also work with Microsoft’s Visual C++, C#, and Visual Basic .NET. And the software also will work with older versions of Visual Basic, Borland’s Delphi, and LabView. A “matrix” of sample code lets engineers choose examples that show how the tools work with a wide range of programming languages and image-acquisition hardware in many types of applications.

When it comes time to verify that a machine-vision setup works properly, engineers can “shoot” test images with a digital camera and use the Foundation Package’s emulator to process the test images. The software “sees” the images as if they have come from a live camera. If system requirements change, engineers can always run through the images again to ensure their revised code executes properly and does not adversely affect other portions of their application.