Keep your digits significant

How you express a measurement affects its meaning.

Brad Thompson, Contributing Technical Editor -- Test & Measurement World, 3/1/2001

Instruments such as this DMM display as many as 5½ digits of resolution. (Courtesy of Keithley Instruments.)

When you count apples, oranges, or transistors, you use whole numbers that yield a precise and accurate count. When you take an electrical measurement, though, you use an instrument to measure an unknown, infinitely divisible quantity. The number of digits you have in an instrument and the instrument’s uncertainty play significant roles in your measurement. Use your measurements in calculations, and you can have more digits than make sense. You have to apply rounding techniques and use the proper notation to get meaningful numbers from your test equipment.

A number’s rightmost digit indicates its precision. In instrumentation, you probably use the term “resolution” instead of “precision.” Resolution refers to a measurement’s “granularity”—the higher the resolution, the smaller the changes in a measurement your instrument can detect.

Suppose you have three DMMs, one with 3½ digits (0–1999), one with 4½ digits (0–19999), and one with 5½ digits (0–199999). If you measure the same voltage with each meter, the 3½ -digit DMM may read 24.8 V, the 4½ -digit DMM may read 24.83 V, and the 5½ -digit DMM may read 24.833 V. If you assume the meters don’t contribute errors to the measurements, then the 5½ -digit meter is both the most precise and the most accurate, because its uncertainty is 10 times smaller than the 4½ -digit meter and 100 times smaller than the 3½ -digit meter.

Errors in readings

Just because an instrument can display a small change in a signal doesn’t mean the displayed value reflects reality. All measurements carry an uncertainty. Some of that uncertainty comes simply from the uncertainty in the rightmost digit’s resolution while the rest comes from errors introduced by the measuring equipment’s circuits. Manufacturers use different methods for expressing uncertainty, usually called accuracy in data sheets. These methods include a percentage of full scale, a percentage of full scale plus a percentage of reading, and a percentage of full scale plus a fixed number of display counts.

Take, for example, a 4½ -digit DMM set to its 20-V range—the meter’s display can read from 0.0000 to 19.999. Assume the following specifications.

• Offset: ±2 counts (±0.01% of 20-V full-scale reading, or 2 mV).
• Quantization error: ±1 count. (At a nominal reading of 5, the last digit may display 4, 5, or 6.)
• Linearity: ±1 count (±0.005% of 20-V full-scale reading, or 1 mV).
• Temperature coefficient: ±3 counts at 25°C.

At 25°C, this DMM’s total uncertainty is ±7 counts, or ±0.035% of full scale (7/20,000). If you connect the meter to a 10.0000-V calibration standard, then any meter reading between 9.993 V and 10.007 V falls within the uncertainty specs of the meter.

Errors’ effects

Just making measurements doesn’t always give you the result you need. Often, you must use your measurements in calculations. Assume you need to make a single measurement and double the value to convert the voltage to an engineering unit. For example, (2.640 V ±0.007 V) + (2.640 V ±0.007 V) = 5.280 V ±0.014 V. Double a number, and you double its uncertainty, too.

To compute the error of a product or quotient of two or more measurements, you calculate the square root of the sum of the squares of each measurement’s percentage error. Power, the product of voltage and current, is one such calculation. Assume you want to calculate the power from 10.017 V and 11.287 A. If you use a calculator, you’ll get 113.061879 W.

Assume that the meter measuring the voltage has an error of 0.06% error and the meter measuring the current has 0.035% of error. Now, calculate the percentage of error:

Applying the percentage error to the power calculation, you can calculate 113.061879 W * 0.06946% = 0.07853278 W. The actual value, therefore, falls in the range of 113.061879 W ±0.07853278 W. You’re now lost in a sea of meaningless digits. Although this calculated range is mathematically correct, the rightmost digits promise far greater precision than most instruments deliver. You should round your results to eliminate the meaningless digits.

Significant digits

In a sum of several numbers, the total must have no more significant digits than the number with the fewest significant digits. In the following example, two significant digits in the first number render the extra digits in the second and third numbers uncertain:

22.12 + 11.123 + 10.1211 = 43.3641

You must round the result to four significant digits, so the result becomes 43.36. (Ref. 1). Even after you truncate a value to the fewest significant digits represented in a calculation, you may need to apply rounding to best represent a value.

Rules for rounding

You can apply a few rules to bring a calculation’s number of digits to within reason:

• If the last digit reads between 0 and 4, add nothing (12.522 rounds to 12.52).
• If the last digit reads from 5 to 9, add 1 to the next digit to the left (12.527 rounds to 12.53).
• If a measurement’s uncertainty doesn’t warrant more digits, you can round off the additional digits. You might, for example, round a reading of 4877 to 4880, 4900 or 5000, depending on how you’ll use the value. If you need a realistic value to compare to another value, then you might need to use 4877. If you need an estimate of power, for example, rounding up to 5000 might be fine.

Apply the above rules to the power calculation. Beginning with the 10.017-V measurement, consider how many significant figures you should carry through your calculations and how many digits to round off the final result. You can assume that the leftmost two digits (10.xxx) are solid and stable. The first digit to the right of the decimal point (xx.0xx) is probably pretty good, too.

The instrument’s last-digit (xx.xx0) error of ±7 counts also renders the next rightmost digit (xx.x0x) somewhat suspect. Thus, depending on the sign of the error, the measured voltage might fall anywhere within the range between 10.010 V and 10.024 V.

The product of 10.017 V and 11.287 A, 113.061879 W, shouldn’t contain more significant digits than its components, so it rounds to 113.06 W. You can also compute the wattage’s percentage error and round it from 0.06946% to 0.07%. Calculate the power’s uncertainty as 0.079142 W, which in turn rounds to 0.079 W (79 mW). The circuit in question consumes between 113.139 and 112.981 W which rounds to 113.14 W and 112.98 W, respectively.

If you use a spreadsheet such as Excel or a symbolic-mathematics program such as Mathcad for routine calculations, you need to consider how the program uses significant figures and rounds numbers. Excel features a ROUND(number,num_digits) option and an extensive collection of cell-format capabilities, while Mathcad offers a round(x,n) function that rounds x to n decimal places. You can specify local or global format of precision and number of digits in a Mathcad worksheet.

When you use measurement results in calculations, you may find scientific notation helpful. Scientific notation helps you identify significant figures in numbers. For example, 0.123 (1.23 x 10^-1) has three significant figures, as does 0.0123 (1.23 x 10^-2) and 123 (1.23 x 10²), while 123.0 (1.230 x 10² ) has four significant figures.

In the expressions above, I used scientific notation. Most engineers use a form of scientific notation called “Engineering Notation.” You probably express electrical quantities as milliamps or megahertz. In engineering notation, you limit the exponents to multiples of three and often add a prefix to the units to represent those multiples of three. There’s nothing wrong with expressing a current as 1.058 x 10^-2 A, but as an engineer, you’ll probably want to change it to 10.58 mA (10.58 x 10^-3 A). T&MW

Reference

1. Myers, Eric, “The Rules for ‘Sig Figs,’” University of Michigan, Winter 1999. feynman.physics.lsa.umich.edu/~myers/140/notes/SigFigs.html.  

For more information

Hosni, M.H., H.W. Coleman, and W.G. Steele, “Application of Mathcad Software in Performing Uncertainty Analysis Calculations to Facilitate Laboratory Instruction,” Kansas State University/University of Alabama in Huntsville/Mississippi State University. www.mathsoft.com/mathcad/library/apps/asee/. Editor's Note 10/24/03: This page is no longer available.

Morgan, Stephen L., “Tutorial on the Use of Significant Figures,” University of South Carolina. www.chem.sc.edu/faculty/morgan/sigfigs/. Editor's Note 10/24/03: This page has moved: http://www.chem.sc.edu/analytical/sigfigs/.

Brad Thompson has been writing for Test & Measurement World since 1986. Currently, he serves as a Contributing Technical Editor and works as an independent electronics consultant and writer. E-mail : brad@tmworld.com.