Test executives keep programs in line

Scalability and reuse are key buzzwords in the electronics industry as companies strive to make the most of limited resources in today's economy. Scalability and reuse apply to your company's hardware, software, and brainpower assets. For the production-test environment, test executives (TEs) can help you make the most of all these resources. TEs can help you reuse code in future applications, they can help you maximize test throughput with a minimum of test hardware, and they can help make your programmers more productive.

Figure 1. Project management tools that facilitate collaborative test development and sequencers that facilitate parallel production testing are key features of test executives. Courtesy of National Instruments.

Figure 2. This test-executive display includes a test-plan definition (left), action definition (center), and test output listing (right). Courtesy of Agilent Technologies.

There's nothing magical about a test executive—you could write your own test-executive functions using any graphical or text-based programming language. But in the spirit of reusability, you might consider making use of commercial test executives, whose authors have combined into low-cost packages a multitude of functions that could take many programmer-months to write.

Furthermore, test executives take care of chores that might be of little interest to you. If your interest is test and measurement of electronic products, you might be happy to farm out tasks like revision tracking, report generation, project management, and corporate network interfacing to a test executive so you can spend your time optimizing the electrical tests you'll apply to your DUTs. The TE might be the only executive in your company ready and willing to take on such grunt work.

Two major commercial test-executive releases have appeared since Test & Measurement World's last look at the topic (Ref. 1): Agilent Technologies' Test Exec SL and National Instruments' TestStand 2.0. Both have a heritage: TestStand 2.0 emerged from the original TestStand release, adding enhancements that facilitate parallel test and project management. Test Exec SL is the commercial, stand-alone version of a test executive that's been bundled with some of Agilent's functional testers since 1995.

Both TEs can work with a variety of programming languages, including Agilent VEE, National Instruments LabView, and various Microsoft visual-programming environments. They also work with a variety of test hardware. Test Exec SL, of course, still works with Agilent's functional test systems, but you can also make use of it as an integral part of a test system provided by a third party, or you can buy it (a developer's kit costs $2995; run-time versions are $550 each) and develop your own test system around a variety of modular instruments, such as IEEE 488 benchtop instruments or CompactPCI card-based instruments. Similarly, you can have National Instruments or a third party provide you with turnkey test hardware configured around TestStand 2.0, or you can buy TestStand 2.0 (also $2995 for the developer's version) and develop your own system around IVI-compliant (Ref. 2) or other modular instruments.

Test sequencers are at the heart of any test executive. The sequencers aim to smooth all aspects of test-program generation and use, from original program development though test-program reuse and on to run-time test-program execution on the production floor.

At the development stage, a TE imposes a structure and common interface that can help test programs—and test programmers—work together. You might believe that structure could cramp your programming style, and indeed, TEs aren't the answer to every test project. As for any programming project, a single, simple, one-level approach might be the most efficient way to solve a problem. Yet, a simple program can expand into an unstructured, difficult-to-document, and impossible-to-maintain nightmare if some form of external control isn't imposed.

It's difficult to determine the exact point at which a TE becomes worth the overhead it imposes. One observer, careful to emphasize that no rule-of-thumb exists, suggests that if there were a rule of thumb, it would be something like this: If you plan on devoting one programmer-month to your project and you plan to deploy $20,000 worth of test equipment as part of the project, then you need a TE. He cautions, though, that there are many more variables than lines of code and dollars worth of test equipment.

For example, a stand-alone test program that you cobble together in a week might evolve into something you and other programmers ultimately spend many months on, adding more and more functions—and bugs. Had a TE imposed its disciplined approach on your initial one-week project, your effort could have become a bug-free, robust, reusable software component instead of the beginnings of a morass of unstructured code that neither you nor your colleagues fully understand.

Your hardware deployment can similarly grow. A single test system might satisfy your initial production needs, but as demand for your product grows, you might have to deploy additional, identical test systems, or you may expand your initial system to accommodate multisite testing.

If your throughput demands mandate multisite testing, you'll have several approaches from which to choose. National Instruments defines two multisite approaches for TestStand: a batch test mode, in which several units undergo identical tests simultaneously, and an asynchronous mode, in which tests proceed in parallel on multiple DUTs but for which specific test sequences for each DUT can vary to make the most efficient use of available instruments and processing power. Agilent Technologies breaks multisite test choices down into three basic categories (Ref. 3):

• Sequential test is suitable for simple cases where the time required to load DUTs into a test fixture is relatively long compared to actual electrical test times. In such cases, an operator or automated handler loads multiple DUTs onto the test fixture; the tester then applies test signals and monitors results. After tests of one DUT complete, the operator or an automated switching system must reroute test leads to the next DUT. When all DUTs are tested, the DUTs are unloaded from the tester and the next set of DUTs are loaded.
• Interleaved synchronous test minimizes test-system hardware requirements and speeds test. In such a scheme, a signal source might be programmed to generate a particular arbitrary waveform. The source then applies the waveform to each of several loaded DUTs in turn. This approach would be particularly useful if instrument programming time is long compared to the length of the test requiring that waveform—the instrument need be programmed only once per set of DUTs, instead of once for each DUT, as would be the case for pure sequential test.
• Asynchronous test minimizes test system costs and improves test speed when instruments must be dedicated to a particular UUT for long periods. In this approach, an independent thread running on the test system's host computer allocates test instruments among multiple DUTs, just as a print spooler can allocate networked printers among multiple desktop PCs.

Ref. 4 provides more detail on asynchronous test. Asynchronous tests can be relatively easy to write—you basically structure your test program to test a single DUT—but only if you make use of a TE to handle the synchronization steps required to transparently manage test resources. The necessary functions will be familiar to anyone who has programmed in a multiprocessing environment. One such function is mutex (for MUTually EXclusive), a function that locks a resource to a particular process or thread. In the case of a TE, instruments as well as processors are the resources being shared among seemingly independent and simple programs. Ref. 5 describes the synchronization steps that TestStand 2.0 employs to share test resources in asynchronous-test implementations.

Finally, a TE not only can help you develop and run your test applications, it also can tell you how efficiently you are making use of your test resources. Test Exec SL, for example, includes a profiler that can help you determine which resources aren't used efficiently. With the results from such a profile, you might determine that a particular expensive spectrum analyzer is sitting idle for long periods of time while cheaper instruments complete their tasks. You might then determine that purchasing extras of some of the cheaper instruments might help you more efficiently use the more expensive instrument.

A TE, like any executive, might occasionally be looking over your shoulder to track your efforts. You can be assured, though, that the TE is only striving to help improve your productivity—from test development through production test.