10 Questions You Must Ask Before Buying Data-Analysis Software

1) What do I need?
Before you can evaluate data-analysis software, you must carefully describe your requirements. You must understand very clearly what type of data you have, what you want to measure, and what types of results you want.

Take a formal approach to establishing your needs by putting your ideas on paper. Then, you can set up a more polished plan to present to the vendors

Make a list of the types of data-analysis tasks you want the software to performÑbasic averaging and graphing through statistical modeling, spectrum analysis, and so on. Determine how much information you want to analyze and what form (not format) it is in: that is, scalars, vectors, matrices, complex numbers, and so on. You will need to find software that can handle the quantities and types of data you have.

Don't underestimate the amount of data you have to process. You may require software that can handle a thousand values now, but future applications may generate tens of thousands of values simultaneously on several channels. Specify for today's needs, but always consider the future.

Also keep in mind the needs of other people who will use the software. If technicians will use the software to analyze SPC data, you must find software with easy-to-understand menu choices as well as with graphs and charts appear that are easy to view and print. If other engineers will use the data-analysis software, you will need software that can quickly switch between various setups.

2) How does the software handle data?
The software should store and retrieve data using industry-standard formats. These formats include text files that use a quote-comma sequence or tab characters as delimiters as well as file formats used by databases (Access, dBase, Paradox, and others), spreadsheets (Lotus, Quattro, Excel, and others), and other types of programs. If software uses a proprietary or nonstandard format to save data, you may find it impossible to exchange data with other applications.

If you need to archive an extensive amount of data, make sure the software can save data in a binary format that takes up less disk space than the formats mentioned above.

The file or data formats should be compatible with those already in use throughout your company. You don't want to spend time processing information that you can't readily share with others. And you may need to analyze data stored somewhere on the corporate network, so any new data-analysis software should accept data in the format your company now uses.

If you plan to acquire data from an instrument or other device, you may want to place the data directly into your data-analysis software. Look for analysis software that can "connect" to standard hardware devices such as a serial port or a data-acquisition system.

Make sure the software will work with your computer or that you can easily modify your computer to run it. You may need to add memory to your computer, or you may need a faster computer to run the software efficiently. Be sure the software supports network operations if you want to exchange data with other users. If you plan to transfer data to and from a database, be sure the data-analysis software comes with drivers for SQL databases: Oracle, Microsoft, Informix, and others.

You may want to do more than save and retrieve lists of values, though. To save the results of graphing operations in print files, the software must support standard graphic file formats such as Postscript, bit map, Windows Metafile, HPGL, TIF, and others.

3) How easy is it to set up and use data-analysis software?
After you load the software into your computer from disks or from a CD-ROM, it should be ready to use. You shouldn't have to spend time setting up initial conditions or writing programs. The software should meet your needs right out of the box.

As soon as you start the program, it should let you import a data file and start trying out data-analysis tasks. If you would rather enter data from the keyboard, the program should provide a spreadsheet-like data-entry form that you can use. Data-analysis software also should let you quickly import data from a spreadsheet such as Excel

It can take time to learn how to manipulate legends and scales on graphs and how to move data to and from other application programs. But you should quickly master the elementary operations of moving data to and from files and of performing simple manipulations on the data. If a software package requires you to puzzle through many menus and steps to run averaging, mean, standard-deviation, and other tasks, move on to another software package.

4) How do I test the software?
First of all, don't rely on canned demonstrations. These demonstrations may highlight the basic capabilities of data-analysis software, but they may concentrate on features and analysis tasks that you won't use. If you have access to the software and you should be sure to use it to manipulate real data that closely approximate the information you want to analyze and plot.

Your testing must ensure that the software does what it is supposed to do. Key tests involve checking correctness and causes of errors. The software should produce the correct results after it processes your data. Some data may cause errors, and you need to know how the software handles these special conditions. For example, you need to know what happens when a set of data includes no-data values at several points. "Missing-data" can cause some programs to produce erroneous results--or to try to fill in the missing data with zeros. You also must determine how the software handles an overflow, a divide-by-zero operation, and other limiting conditions that could cause errors

Here are several tests that manufacturers recommend for testing data-analysis software:

1. Create a set of data and then create multiple types of graphs and put the graphs all on one page. The software should handle several types of graphs simultaneously without any problems
2. Perform the following sequence of operations:

• Import a large data file of more than 100,000 points.
• Display the data.
• Run a data-analysis operation, perhaps a fast Fourier transform (FFT)
• Plot the data after adjusting axes, legends, and scale.
• Rotate and switch axes.
• Print the plot
• Save the results to disk or,
• Import a set of data.
• Graph the data
• Perform a curve fit
• Plot the curve-fit graph.
• Print a document that includes the results.

Set up the software to perform the five steps in the second list automatically. After all, you don't want to have to repeat them over and over for repetitive processing.

5) What about documentation and support?
Obviously the documentation for software must be complete. "Complete" means that the documentation includes not only a user's manual but also reference information that carefully describes menu choices and operations. These days, much of this information comes in help files, but I prefer books in which I can highlight sections, underline, and add sticky notes as bookmarks.

Look for well-written manuals that actually guide you through operations one step at a time. Instructions must be clear, and a book on data analysis should include examples and plenty of helpful diagrams. Check a manual's index. If all the keywords fit on only a couple of pages, you'll find it a challenge to look up cryptic commands.

Be sure the company offers technical support. Find out if technical-support people will answer your questions even after the company replaces your product with a newer version. One company, Synergy Software, provides free support for the life of the product

Look at the company's Web site to find out if it has a discussion forum of users or if it posts technical-support information for all users. In addition to finding the answers to your questions, you also may learn out about neat features and "tricks" not explained in the manuals.

6) Can I program the software?
The basic software should perform the operations you need without your having to resort to programming. If you expect to customize the software for some reason, be sure to read the manual to see how much you can modify the operations.

Flexible data-analysis software should let you record macros that will automate some tasks. More sophisticated software may allow for programming scripts that act like programs. You shouldn't have to write scripts unless you need something special from the software. Scripts and macros come in handy, though, when you have repetitive data-analysis tasks that require dozens of menu selections.

Some software comes with its own programming language. For example, DADiSP provide a C-like programming language, Grapher 2.0 from Golden Software includes a Visual Basic-compatible scripting language, and even Excel comes with a Visual Basic-like language.

7) How much do I need to know about data-analysis?
Would you design a circuit without knowing the difference between a 100-V and a 10-V signal? Well, you shouldn't take on a data-analysis task unless you know what you're doing. If you want to do some averaging and basic statistical operations, you can probably get by without difficulty. If you want to do curve fitting, perform a Fourier transform on your data, or get into analysis of variance calculations, however, you need to brush up on what happens to the data. Frankly, the more you know about data analysis and statistics, the better off you are.

Anyone can make menu choices, but if you don't know the underlying data-analysis principles, you may not know whether you're using the appropriate analysis operations. For example, if you need to examine trends in your data, do you use a least-squares, fit, simple averages, moving averages, or some other math operation? All these operations yield information about trends, but you need to know how each operation treats data and what the results represent before you can get useful results.

Even when you choose the appropriate operations to process your data, you still need a feeling for the type of results you'll get. For example, if you run a Fourier analysis on your data, you should have an idea of what the results will look like. If the results differ markedly from your expectations, look to see if you set up your sampling rates, windowing, and padding operations properly. And you need to check your raw data, too. Perhaps the sensor you're monitoring never got hooked up, or perhaps it's hooked up improperly. If you don't have a feeling for what the results should look like, may never know that you have erroneous data to start with. (Of course you should check the raw data, too, but how many of us do that?)

But don't discount results because they look unusual. If you have checked and rechecked your data, your instrument setup, and the processing steps, the analysis results may be real. You may have discovered something new about the phenomenon you're investigating

8) How flexible is the graphic output?
When you plan to produce many charts and graphs, look for software that gives you the ability to easily change formats, add annotations, change labels, change line styles, change fills, and so forth. The software shouldn't force you into using just a few inflexible styles for printing visual information. Because color printers are now common, look for software that gives you a good palette of colors for your charts and graphs.

The software also should let you easily move graphic information from application to application. For example, moving a chart into a word processor such as Word or WordPerfect should require only a few mouse clicks

Look for software that can create the types of plots you need. For plotting data spanning over several decades, you may need logarithmic axes. Can the software draw semilog plots using either the x-axis or the y-axis as log scale? Can it produce log-log plots? You must have an idea of how you want to present your information to know whether or not the software can plot it properly for you.

9) Can I get a guarantee?Unfortunately, few software suppliers offer a money-back guarantee unless you return unopened software. But it doesn't hurt to ask. At the least, suppliers should offer a free 30-day trial period so you can test the software on your computer and network and use the software to analyze real data.

If a company won't give you a 30-day trial, it should provide a demonstration program. The demo should work like the real program, but it may not let you try all its features, such as saving files or printing graphs. The demo should let you try the program with your own data. Make sure the demo lets you try the features you'll use when you analyze data for real. Watch out for canned slide-show demos. They are useless for evaluating software.

10) How can I save money?
First of all, don't buy software with a lot of features you don't need. You may find software for a few hundred dollars that does what you want and has enough capabilities for future needs. The packages we surveyed cost between a few hundred dollars and $10,000. Second, always compare products based on factors such as price, technical support, documentation, and ease of use.

Ask the supplier how much it will cost to upgrade when the next version of the software reaches the market. You may find it's less expensive to buy a $1500 package that lets you upgrade for $150 than to buy a package for $500 now that you can upgrade only by purchasing a complete package. Also, before you buy any software upgrades, be sure you'll use the added features and capabilities. If you won't, why upgrade?

Compared to the market for word processor software, the market for data-analysis software is small. Still, you may find prices below list price by searching software outlets on the Web or by working with resellers who specialize in software for engineering and scientific applications. T&MW