Accreditation brings unforeseen benefits
Martin Rowe, Senior Technical Editor -- Test & Measurement World, 3/21/2007 6:40:00 AM
Audio Precision has been manufacturing audio analyzers since 1984. The analyzers require verification and calibration prior to shipping. Over the last few years, company chairman and co-founder Bruce Hofer began to see the value of having the company's calibration lab achieve accreditation. In January 2007, the company announced that it has achieved accreditation from the American Association of Laboratory Accreditation (A2LA) under ISO/IEC 17025:2005.
Attaining accreditation took several years and thousands of work hours, but now the company can prove measurement traceability to national standards, something that many customers require. Although the process was tedious, it opened many eyes within the company. It also led to engineering changes that improved the company's products. To find out more, I spoke with Hofer by telephone from his office in Beaverton, OR.
Q: What changes did you have to make to your calibration processes to attain accreditation?
A: We had to make several changes, but first we had to convince management of the benefits of accreditation. Although I'm one of the company founders, I'm first and foremost an engineer so I had to convince other members of the management team that the project was worthwhile.
Q: What was the problem with management?
A: It's difficult for some managers to understand the benefits of calibration. Some people see it as a cost—mostly in new equipment—with little or no benefit to the company. The payback is often intangible because you can't charge your customers more for accredited calibration. They expect your products to meet published specifications. Still, our customers were demanding that we prove traceability to national standards, NIST in our case. Some of our Japanese customers were performing their own calibration on new equipment to prove traceability.
Q: Bruce, you're not alone. Engineers have told me for years about how they've had to work hard to convince management of the need for traceability. It's not an easy sell.
A: It's easy for a manager to say "Yes, I'm all for calibration." But, there's a big step between saying you're for calibration and quality and actually budgeting the capital to implement a program.
| Have you worked to convince management of the need for traceable measurements only to get resistance? Post your comments. |
A: We had to make sure that everyone understood what the word "calibration" means and break down its misuses. Calibration used to mean the process of adjusting a product to bring it as close to nominal performance as possible. Today, calibration means that you produce a certificate stating that the product was verified using proper procedures and with test equipment that's traceable to a national standard.
Q: Did you have to obtain traceable test equipment?
A: Yes. We purchased a Fluke multifunction calibrator and hygro-thermometer, an Agilent 8 1/2-digit DMM and frequency counter, and a Tektronix oscilloscope. We use them as references.
Q: What technical changes did you have to implement?
A: First, we had to review what we wanted in our data sheets. A data sheet is a guarantee how you want your product to perform. That is, specification X plus or minus Y percent. You must demonstrate conformance to your specifications through a formal process and traceable equipment. Doing so will prove that your measurement techniques and equipment are credible and reproducible.
You then have to decide which measurements you can make directly and which you must derive. Some things involve two or more steps. One example in audio is flatness, the degree to which a device's frequency response may deviate from ideal.
Q: How do you make a flatness measurement?
A: You start with a reference frequency and measure amplitude. Then, you measure amplitude at other frequencies and compare your results to the reference measurement. So, when we say "flatness to ±0.1 dB," we're referring to the difference from a reference amplitude measurement at several points in the spectrum, typically from 20 Hz to 20 kHz. You have to decide which points to make your measurements. You should make the measurements at the most problematic points. If you pass at those points, you'll pass at the others.
Q: Do you have to state the uncertainty of your measurements?
A: For accreditation, you must include an estimate of your uncertainty for each measurement. Stating uncertainty is the most difficult part of the accreditation process. You have to account for the uncertainty added by your test equipment, fixtures, cables, and measurement techniques.
Estimating measurement uncertainty is a grueling process. You have to think about every step. For example, you must consider the resistance in a cable. Many people assume that a cable has no resistance—zero ohms. There's always some resistance and you need to measure it because it contributes to measurement errors, or uncertainty. You have to add all possible errors into your calculations. The easy ones come from data sheets of your test equipment (DMM or calibrator). Others, you have to measure or estimate.
Q: Can you give an example of the calculations?
A: In our flagship product [Model 2700], there are about 48 steps and 400 data points we need to take. For each point, we had to create a spreadsheet showing the calculation of worst-case uncertainty.
Q: How did you learn about uncertainty calculations?
A: We hired a consultant about two years ago, a former cal lab manager at Fluke who's now retired. He's also an A2LA auditor, but we agreed that he shouldn't participate in our A2LA audit for ethical reasons.
We learned that when making uncertainty calculations, you must consider measurement repeatability, not just specified performance. Your device under test isn't perfectly stable, even in a cal lab where temperature may be controlled to within 1°C.
When you make a measurement, repeat it at least six times, preferably ten times. Then, compute the standard deviation. It's usually not zero. Some of that deviation may be due to the resolution of your instruments, but you'll get some deviation from the device under test itself. We didn't realize that at first.
Q: What did those variations tell you about your products?
A: We discovered that certain ranges showed more variation than others. We traced it back to resistors that needed tighter temperature coefficients. That led to some minor circuit changes. So, we were able to improve product quality because of the audit.
We also learned that measurement repeatability can sometimes swamp out all other factors in your uncertainty analysis by one to two orders of magnitude. Auditors like to see how you deal with repeatability in your uncertainty calculations.
Q: How much work did you put into preparing for the A2LA audit?
A: We've logged nearly four work years over a three-year period to achieve accreditation. That's about 6000 to 8000 hours of work. I put in between 500 and 1000 work hours just on the uncertainty calculations. That was for one product, but for additional products, the amount of effort drops significantly. That's because we now have the procedures in place.
Q: How has accreditation changed your calibration process for outgoing products?
A: Every product goes through our calibration station at the end of the production cycle. We can not only certify that outgoing products are calibrated to traceable standards, but we can also check our own in-house test equipment. We've always done the calibration, but now we have documented proof of traceability and uncertainty.
Q: Would you say that attaining accreditation was worthwhile?
A: Yes. Besides having proof of traceability, we learned a lot. I personally enjoyed the rigor of developing uncertainty calculations and we learned ways to improve our products.
Does what Hofer said sound familiar? Have you worked to convince management of the need for traceable measurements only to get resistance? Post your comments to "Rowe's and columns."
