Test-system development: Do everything first

In specifying a test system's functions, start with the measurements, define the criteria, and justify each test. "As a test engineer," noted Lewis, "you must ask, 'What value does this test add to the product?'" Blackford added that for military and aerospace electronics, you should "test it like you fly it and fly it like you test it. Don't test it for parameters it wasn't designed to meet and don't perform unnecessary tests. Test for every parameter that you will use."

Once you define the tests, you can order equipment. ("Equipment compromises," below, explains how approved supplier lists and legacy equipment can hinder test-system performance.) As Leuschen noted, you shouldn't wait until a month before a project is due to order test equipment: It may take weeks or months to arrive. While waiting for the equipment, hardware engineers can install cables and build test fixtures.The system integrators at the table also explained that they often use older or rental equipment to keep system development moving while the permanent equipment is on order.

The system integrators also said that when developing test systems, they perform many checks on cables, connections, and instruments. They never apply power to a DUT before checking all connections and signals. For example, they use DMMs to passively check connections for continuity. If a connector or cable will supply power, the integrators will use a DMM to check that the proper voltage is available at the proper test points. When a system needs to measure resistance, the integrators use a dummy load rather than a real DUT to check it.

In some cases, the system integrators use a test-system checker to bring a system online, and their customers use the same checker to periodically verify system performance. Test-system checkers range from a simple loopback box for checking signals to an automated rack of signal sources and measuring instruments.

Figure 1.  An automated test-system checker can keep a production test system calibrated and running.

Automated test-system checkers (Figure 1) are expensive, but they can put test engineers and managers at ease. Automated checkers can reduce test-system downtime by easing troubleshooting and verifying measurement accuracy. They may even provide equipment calibration. If you need several test systems, you may be more easily able to justify the cost of an automated system checker.

"Whether your test-system's checker is manual or automated, you need a verification procedure to make sure that the test-system in question works," noted Lewis. "Verification procedures are also useful once the test system is in the factory. If you suspect a problem, how do you debug the test system? Is the problem with the product or with the tester?" Blackford provided the answer when he said, "If the last three units failed, it's the tester. Management says so."

How often you should run verification procedures depends on the cost and complexity of the UUT. One-of-a-kind UUTs that cost thousands of dollars may require you to run a verification procedure before each UUT. In low-volume, high-mix production, a verification may be needed each time you change UUTs. In high-volume production, a verification interval may be once per shift or once per day.

While hardware engineers connect instruments, check connections, and make measurements, software engineers define a software architecture, develop the user interface, write automation code, and design reports. They may also develop a "simulation mode" for testing functions before the instruments arrive.

Before developing a user interface, the software engineers on the panel consult with the engineers who will maintain the system, the engineering or production managers (depending on the application), and the technicians who will use the system every day.

"When developing the user interface," said Kirchner, "don't under develop or over develop it. Keep only the necessary information on the screen, and make the interface so that the user can easily get the extra piece of information. That requires a lot of interaction and getting to know the customer."

Lewis added, "I see applications where there's a lot of information on the screen, but no one really does anything with the information. The information may become so small as to not be useable."

"We've developed a set of guidelines that comes from our experience at developing test systems." said Gamez, "For example, if the operator on the manufacturing floor is far away from the screen, then you need larger fonts than if he or she is close to the screen."

Figure 2.  Software engineers develop the user interface and logic layer for an automated tester before developing the sequence layer.

The logic layer contains code that automates instrument functions by sending instructions to instrument drivers. Gamez prefers to write code for this layer before writing sequence code. He uses the test-system specification and the list of instruments as a guide to developing his code. He works with hardware engineers to write code because they're more familiar with the measurements and the test equipment than he is. If an instrument is on order and a substitute isn't available, Gamez will develop code that simulates the instrument to keep the project moving forward.

Software engineers should test each code module. "We write small sections of code, and test each one individually," said Gamez, "That way, at any given point in the process, we have some working software."

The sequence layer lets test engineers set up a test system's sequence of events. Often, test sequences will change depending on the functions of a UUT. Code in the sequence layer invokes sections of the logic-layer code depending on the tests required and the order they need to run. Gamez noted that engineers at VI Technology may use a commercial off-the-shelf test executive as a sequence layer or they may develop one with a programming language, depending on the customer needs.

A sequence layer must do more than just call logic-layer code. It must collect data and store it in a format that provides for reports. Unfortunately, data presentation and reports are often afterthoughts. Planning for data presentation and reports lets you make the most of the oceans of test data that a system can generate.

The system integrators also noted that a test system needs some way to deal with errors and faults. They require their customers to plan for error-checking features. Lewis cited that in extreme cases, as much as 75% of software development time may be devoted to error handling.

Gamez cited cases where the amount of error checking depends on the value of the UUT. If an error can cause damage to an expensive UUT, then make sure the software can catch it. Blackford recommended that test engineers check every possible path and software function. But he noted, "If you have an 850-page test-requirements document and a seven-rack test set, you'll probably miss something that a technician will uncover at 2 a.m."

When planning a test system, the system integrators on the panel look not only at what it takes to get a test system running but also at ongoing support. They may develop diagnostic software that uses a system's instruments to check the system or that works in conjunction with a manual or automated system checker.

They also consider the skill set of the engineers who will maintain the system, and they use a programming language that in-house engineers can modify and support. "If the customer is tied to a particular language," said Leuschen, "then he will insist on it, even if we think it's the wrong tool." But they will advise against using an outdated language that will hinder upgrades.

Developing an automated tester takes planning, so plan for each phase of the system development at the start of the project. Early planning and execution reduces risk and provides for a functional and maintainable test system.