Home>Characterization Test Center>Test How-To

Keeping the Navy calibrated

By Martin Rowe, Senior Technical Editor- September 1, 2007

Norco, CA—When Navy pilots use night-vision goggles, they trust that the goggles will provide the proper sensitivity. When Navy personnel use radios, they expect the radios to operate on the correct frequency. And everyone aboard a Naval ship relies on instrument gauges to accurately indicate quantities such as temperature and pressure.

Arman Hovakemian manages the measurement science department, which provides measurement assessment and calibration procedures to Navy and Marine Corps calibration labs.

To ensure that these and other weapons systems perform as expected, defense contractors develop specialized test and measurement equipment. But who tests the testers?

The answer is the Navy itself. Technicians on land-based and ship-based calibration labs test and calibrate the Navy’s more than 1.7 million pieces of test equipment, performing more than 525,000 calibrations per year.

The Navy has more than 5000 active calibration procedures in use around the world. All calibration procedures (manual and automated), test-equipment evaluations, and measurement analysis, as well as some calibration standards, originate in the Measurement Science Department at the Naval Surface Warfare Center, Corona Division (NSWC Corona, www.corona.navy.mil).

Headed by measurement science director Arman Hovakemian, the Measurement Science Department is the Technical Agent for the Navy’s metrology and calibration (METCAL) program. More than 250 engineers, scientists, and technicians support 409 calibration labs, 242 on ships and 167 on the ground. The work force is entirely civilian. Most of the staff are government employees, but some are Navy contractors. Figure 1 shows the hierarchy of the five divisions that report to Hovakemian:

• Measurement Requirements Assessment (MS10). Assesses measurement requirements for Navy and Marine Corps weapon systems associated with acquisition programs. It also develops calibration support plans.

• METCAL (metrology and calibration) In-Service Assessment (MS20). Assesses measurement reliability and readiness for Navy and Marine Corps test equipment that supports operational systems. Staff members also provide technical support to calibration laboratories and Systems Command.

• Metrology Engineering Assessment (MS30). Develops calibration procedures for test equipment, provides technical requirements for calibration facilities, performs metrology training, and produces specifications for calibration standards. Three technical subgroups focus on DC/low frequency, microwave and electro-optics, and physical and mechanical measurements.

• Advanced Measurement Systems (MS40). Assesses new weapons-system measurement requirements and performs metrology R&D in cooperation with the Air Force and Army. Staff members coordinate with the National Institute of Standards and Technology (NIST) to transfer Navy-designed calibration standards to industry.

• Strategic Systems Evaluation (MS50). Performs assessment and procurement of general-purpose test equipment and calibration standards.

Figure 1. The NSWC Corona Measurement Science Department consists of five divisions that handle all aspects of equipment specifications, calibration procedures, standards, automation, and purchases. SEE LARGER VERSION OF THIS FIGURE.

The need for standards

To the military, “calibrate” means “compare” rather than “adjust.” Navy technicians on ships and on land will verify that a weapons-system tester is within tolerance and adjust it only when it’s out of tolerance.

To perform a calibration, the technicians need calibration standards. When you think of a calibration standard, you probably think of a high-end digital multimeter (DMM) or multifunction calibrator, but the Navy refers to anything used to calibrate anything else as a standard.

Jeff Walden leads engineers who assess the Navy’s metrology and calibration needs.

In many instances, the engineers in the Metrology Engineering Assessment (MS30) group determine that a basic DMM or oscilloscope can become the standard that technicians use to calibrate weapons-systems testers. In cases where the Navy needs a calibration standard with better uncertainty than is available in commercial off-the-shelf (COTS) equipment, the NSWC Corona engineers develop their own standard.

One example involved night-vision goggles. “In the early days of night vision,” said metrology assessment manager Jeff Walden, “there were many challenges. The Navy decreed that night-vision goggles must be tested before each use. Thus, the tester must be accurate. It must emit a known, low level of light that the goggles must detect to pass a test.”

To ensure that the tester emits the proper light level, NSWC Corona engineers developed a photometer that measures the light levels from the tester, and they transferred the design to the contractor that builds the goggles and their testers. The engineers also wrote the procedures that Navy technicians use to calibrate the goggle testers.

Navy pilots rely on infrared cameras to help them find targets. The camera detects spatial frequencies that correlate to temperatures. A camera tester uses a blackbody source that produces a bar pattern. To verify the blackbody source, technicians use an infrared detector developed at NSWC Corona.

Procedural process

When engineers in the Measurement Requirements Assessment (MS10) group determine that a weapons system needs testing, they also determine if the test system needs calibration. If it does, an engineer in Walden’s MS30 group will develop a calibration procedure for a tester. Testers and their calibration procedures must satisfy the technical requirements that MS10 engineers specify. The technical nature of the calibration (electrical, electro-optic, or mechanical) determines which MS30 engineer will write the procedure.

Bob Fritzsche leads the METCAL in-service assessment group, a team of engineers who support instrumentation on ships and on the ground.

Because all Navy and Marine Corps calibration procedures originate at NSWC Corona, technicians around the world calibrate test equipment the same way. “Corona is the organization assigned the task of developing, reviewing, approving, and distributing calibration procedures for the Navy and Marine Corps. Centralizing that function helps to create standardization of format, content, and technical approach across the Department of the Navy.” said METCAL (MS20) manager Bob Fritzsche.

“A test procedure will include all the information that a technician needs to perform a calibration,” added Walden. “That includes calibration standards, logistics, safety precautions, and special facility requirements.”

The Measurement Science and Technology Laboratory, which houses the Measurement Science Department, has a lab that engineers use to evaluate calibration procedures and to develop automated calibration systems. Engineers at NSWC Corona perform few calibrations on test equipment—calibrations take place in the field.

After an engineer writes a calibration procedure, called an “Instrument Calibration Procedure” (ICP) by the Navy, he or she will take the procedure to the appropriate calibration lab to verify the procedure’s integrity. Then, an ICP will go to one of six metrologists, or “subject-matter experts,” for a quality technical (QT) review.

Figure 2. Instrument calibration procedures (ICPs) require several major process steps during their development. Courtesy of NSWC Corona. SEE A LARGER VERSION OF THIS FIGURE ALONG WITH DETAILED FLOWCHARTS OF EACH BOX.

In a QT review, a subject-matter expert looks for details such as test-uncertainty ratios between the equipment under test and its calibration standard. Figure 2 shows the process for developing and verifying an ICP. (See a larger version of Figure 2 and detailed flowcharts of each box.)

Following a QT review, an ICP goes through a practical review. “One of our engineers will bring a procedure to a Navy or Marine facility and perform a bench test on the procedure with one of the laboratory’s technicians,” said Fritzsche. “The engineer will update the procedure to incorporate the inputs from the technician.”

Aviation electronics technician Sheridan Lee calibrates a pair of night-vision goggles aboard a ship. NSWC engineers wrote the calibration procedure that Lee follows.  Courtesy of US Navy.

ICPs also receive a quality formatting (QF) review. ICP process manager Julie Cunavelis explained that Navy contract support personnel edit ICPs for formatting consistency. They correct details such as typing errors, line widths, and spacing. “Following a QF review, someone will perform a final bench test of the ICP,” she said. “Bench tests take place at NSWC Corona or at another Navy lab by someone other than the ICP writer. Even with all those reviews, we still find glitches.”

When an ICP reaches final approval, Cunavelis’ group will convert the written procedure to a PDF version that will be reviewed by someone from the METCAL group. Cunavelis will also give the ICP one more review. When it’s ready, the approved ICP goes onto the monthly CD-ROM update. Each month’s CD-ROM contains all in-service Navy test procedures. Upon receiving a new disk, technicians at each calibration lab destroy the previous edition.

The monthly CD-ROM update contains more than just test procedures. It contains a database of calibration intervals for each piece of test equipment. Calibration intervals may change even when a test procedure doesn’t, so labs always work from the latest CD-ROM.

Internal analysis

To determine calibration intervals, engineers at NSWC Corona constantly receive and analyze calibration results from calibration labs. Because of the vast numbers of Navy test instruments, NSWC Corona engineers have perhaps the most calibration data of any organization in the world. Engineers study the data, usually over a three-year period, and recommend changes to calibration intervals.

If a particular instrument is consistently within tolerance, engineers may lengthen a calibration interval, which reduces the Navy’s cost of ownership. If a particular instrument model is found out of tolerance, however, engineers will look at the impacts that an out-of-tolerance instrument has on testers and ultimately on weapons systems. They may recommend reducing calibration time or they may look for the cause of the condition and correct it, possibly by modifying the test procedure.

Data analysis is just one of the activities that NSWC Corona undertakes to reduce calibration costs. Hovakemian explained that the measurement science department has embarked on an automation and modernization program. “Just because we need something doesn’t mean we get it, so we have to save money,” he said.

Fritzsche emphasized this point when he added, “The Navy needs to spend money to purchase and update its bombs, planes, and ships. Thus, maintenance dollars to support those systems are often hard to find, and funding for calibration, which is essentially the maintenance of the maintenance equipment, is even more scarce.”

Night-vision goggles require testing before each use. NSWC-Corona engineers developed a standard to calibrate night-vision goggle testers. Courtesy of US Navy.

Perhaps the most significant cost-cutting project involves automating selected calibration procedures. Richard Schumacher, an engineer in the Advanced Measurement Systems (MS40) group, demonstrated an automated calibration on an Agilent Technologies function generator and DMM. Navy engineers have selected 30 calibration procedures to automate, based on workload and level of difficulty.

“We automated a DMM calibration first because it was easy,” said Schumacher, “but it’s not a good test. So, we then developed a procedure for one of the most difficult instruments to calibrate, an RF test set, which uses 15 calibration standards. This instrument used to take 12 hours to calibrate by a highly trained technician. Now, a lesser-trained technician can calibrate it in about 3½ hours.”

The automated system, called METBENCH, consists of an embedded computer module that doesn’t have a hard drive. For security reasons, the operating system—a custom version of Linux—and the calibration software reside on a bootable USB thumb drive. Technicians at shore-based labs get the calibration procedure from the Navy’s computer network, while ship-based technicians get procedures from a satellite link. (The monthly CD-ROM contains instructions on how to use the automated procedure.) Calibration results will also go on the USB thumb drive. NSWC Corona engineers deployed the first automated calibration procedures on a ship-based lab.

Schumacher explained that NSWC Corona engineers chose Linux for security, its small core, and no license fees. The computer connects to instruments through a USB-to-IEEE 488 interface.

“We developed a software tool that lets us develop automated calibration procedures without programming,” said Schumacher. “We try to develop a single procedure that can automate the calibration of more than one instrument. We used to develop manual test procedures for each instrument model. Now, we can have one automated procedure for a class of instruments, such as DMMs.” The system supports SCPI-based instruments and pre-SCPI instruments.

To perform a calibration, a technician logs into the system. Technicians have access only to those procedures that they are authorized to perform. The system then polls the instrument bus to find connected instruments, which includes equipment under test and those instruments used as calibration standards. A user log records every command sent to and response received from all instruments. Thus, the Navy can prove that an instrument was calibrated, and it has a time-stamped record of the entire procedure.

Schumacher and his colleagues in the MS40 group are also automating instrument management. A Web-based application lets lab technicians and managers locate any instrument or tester that requires calibration. A bar-code system records information on every person who handles an instrument. A technician’s log-in information determines how much information he or she will see.

Automation can greatly reduce calibration time, provide greater repeatability, and provide a channel for NSWC Corona engineers to retrieve test results. An equipment-modernization program spanning a three-year period should further cut costs.

Replacing equipment

When it is time for the Navy to update its systems and calibration procedures, the MS30 group is responsible for evaluating replacement equipment. Brennan Heglar manages the physical and mechanical group. “We’re looking to replace aging and obsolete calibration standards,” he said. “Electronic standards have a five- to seven-year life while mechanical standards last 15 years or more.”

The engineers look not only to replace standards but also to reduce the number of standards they must support. For example, a lab may use different standards for voltage, current, and frequency. The engineers may try to replace those three standards with one instrument.

“Electronic instruments make many measurements,” said Fritzsche. “Time on the bench counts, so using a standard that produces several signals cuts test time over using multiple standards. Plus, from a logistics and cost perspective, we have fewer calibration standards to support.”

The MS30 engineers base their decisions to replace instruments—whether they’re used for testing weapons systems or used as calibration standards—on calibration and usage data. By looking at which systems use a particular standard, the engineers can determine the impact that a new standard will have on equipment maintenance. They look at recurring costs of existing standards versus the purchase cost and maintenance costs of buying new standards. A replacement instrument must have, at most, a five-year payback period.

Navy engineers will use commercial off-the-shelf (COTS) equipment wherever possible. “COTS is key,” said Heglar. Unfortunately, a replacement instrument isn’t always available. Take, for example, the HP8902A RF measuring receiver. NSWC Corona engineers have to use three instruments to get comparable measuring capabilities. “We’re working with test-equipment manufacturers to develop a replacement instrument,” said Schumacher.

NSWC Corona engineers can also reduce costs by negotiating multiyear buys with equipment manufacturers. “We were able to buy an instrument for $10,000 per unit through a contract rather than pay $70,000 without a contract,” said Walden. “We also combine our buys with those from the Army and the Air Force, which can reduce costs.”

Besides coordinating equipment purchases with the other military services, NSWC Corona engineers confer with their Army and Air Force counterparts to share technical information, including calibration procedures when possible. “We work with other services,” said Hovakemian. “We regularly hold meetings with the Air Force and Army where we look for ways to avoid duplication of calibration procedures or facilities. If the Navy needs a calibration and the nearest lab is run by the Air Force, we’ll use that lab.”

NSWC Corona engineers also participate in technical conferences such as the Measurement Science Conference (www.msc-conf.com) and the NCSLI Conference and Symposium (www.ncsli.org/conference). In fact, Fritzsche served as 2007 Measurement Science Conference president. He and others also work closely with NIST on metrology issues such as traceability to national standards.

Recently, Hovakemian and others in the Measurement Science Department have partnered with two local universities—University of California at Riverside and at Irvine—on educational programs to get engineering students to consider careers in metrology.