Home>Other>News Archive

Do you know your true cost of test? (Guest commentary)

By Richard McDonell, National Instruments- November 2, 2009

Editor's note: This article is part 1 in a series on "Engineering a Competitive Test Strategy." Also see part 2 ("Standardization —what does it really mean?") and part 3 ("How do you treat test like a product?").

Many companies look to their design and manufacturing organizations to maintain their competitive advantages in the marketplace. These firms consistently measure common metrics such as development cost, time to market, throughput, quality, and number of returns in order to drive improvements in product-development schedules and processes. But the role of these companies' test strategies in maintaining their competitive advantage is often less clear. This lack of clarity commonly results from less focus on the measure of test-related processes and metrics compared to design and manufacturing.

To illustrate this point, I encourage you to ask some of your colleagues and those in other departments, "How does your company's test strategy play a role in maintaining your company's competitive advantage?" Most will likely comment in general terms that test serves a vital role in the design and production of your competitive products (as it most certainly does). Very few, however, will likely be able to tell you the actual business impact in terms of measurable test cost, throughput, time to market, or quality as a result of your company's test strategy.

This inability to quantify the impact presents a golden opportunity for you to educate your company, especially during this challenging economic period, on the quantitative value your team's test strategy contributes directly to the company's competitive advantage. Capturing a true measure of your business impact also helps you identify areas that could experience significant improvements, including sizeable cost savings, through strategic investments in your test strategy.

Determine the true cost of test

A fundamental way to assess the business impact of your test organization is to determine your true cost of test. "True" cost of test is emphasized because test cost is often solely attributed to the capital-equipment costs of test systems. While capital cost is a key component, there are many other critical elements that should be included as well.

In fact, the most complete assessment of test cost is one that takes into consideration the following: development test cost, deployment test cost, and operational test cost. The combination of these components represents the total cost of test for a product and a company depending on the scope of the inputs.

Some of the common development-test-cost elements are test-research costs, development-tools costs, development-training costs, development-time costs, and internal-training costs (in the event of a developer being used as an internal training instructor). It is important to note that your organization may have additional development test costs that are not represented in these common elements. You should look for synergies among those outlier costs and account for them through new cost element categories or as individual cost elements within your development test costs.

The goal is to ensure your entire test costs are accounted for properly. The cost elements and categories may vary from project to project and almost always vary from company to company.

Some organizations who are going through or have recently gone through a standardization effort should consider including the development costs of the time spent on research efforts leading up to the definition of their new standard test architecture as part of the cost for test research. One caveat is to be sure to only include the percentage of time per developer allocated to the standardization research since most are not allocated at 100%. This helps to avoid double counting in your development-test-cost calculation.

Deployment test costs are typically the easiest to determine since they most often include the cost of the capital equipment for the test system along with any software deployment costs as the two main elements. There may be additional deployment test costs for your organization, so it is important to carefully examine these deployment costs to be sure they are all accounted for.

Operational test costs commonly consist of a combination of personnel costs, maintenance costs, and facility costs. More specifically, personnel costs typically roll up from the operators and technicians, while maintenance costs reflect the cost of maintaining spares to prevent downtime in the event of a failure or need to remove an instrument for routine calibration.

Floor space costs represent the facilities cost of operating your deployed tester from a building, power, and cooling perspective. Accurately accounting for the cost of floor space is often inadvertently omitted in cost assessments and may require speaking with your facilities department to validate your cost assumptions. In many cases, the cost of floor space is relatively low compared to other test costs—until you run out of it. Then, suddenly, the cost of additional floor space can easily dominate your cost of test due to the expense required to build or rent another facility that meets your manufacturing requirements.

Knowing your true cost of test at the project, department, and company level is an extremely powerful tool for you and your organization to have for representing the true value-add of your test strategy and to help you justify future investments. Similar to the processes and metrics tracked in the design and manufacturing organizations, it allows you to track your test ROI (return on investment), cost per unit tested, annual test costs and savings, payback periods on strategic investments, and the breakdown of capital versus noncapital costs in your test organization.

This information is invaluable when trying to determine which areas of your test strategy to invest further in for the maximum ROI and in which areas you could possibly reallocate resources within your team or across departments. It can also assist you in determining process improvements across departments that can significantly improve your company's competitive advantage by improving efficiency and lowering overall design and manufacturing costs.

Many of the industry-leading electronics companies have discovered the opportunity to further their competitive advantage with an optimized test strategy. The question in some cases is whether they have a true cost of test model to determine their optimal test strategy. Take advantage of this opportunity to be a leader in your organization and help demonstrate how test is playing a major role in furthering your company's competitive advantage.

Richard McDonell is senior group manager of product marketing at National Instruments. richard.mcdonell@ni.com