Design and documentation go hand-in-hand in successful software-development projects

Taniza Afrin, Hughes Network Systems Product Design Group, Germantown, MD -- Test & Measurement World, 9/22/2003

After my five years of experience as an object-oriented programmer followed by two years of experience as a test system engineer with the Product Design Group of Hughes Network Systems, I have come to realize that software developers need to follow certain rules to come up with the best solution for their customers. If you develop software, you’re probably aware of these “rules of thumb,” but you may find it difficult to follow them consistently. If you follow them as I describe here, though, you can ease the pain of developing, deploying, and maintaining any software.

Any software development cycle involves a variety of people: system designers, software developers, users, and many others. A project begins with system designers and software developers working together in a predevelopment phase. Then, two processes—developing the software and documenting the design—run in parallel for any development cycle.

Software development requires careful considerations to ensure reliable, user-friendly, bug-free software. In general, a software developer needs to follow these five steps:

1. Understand the customer's requirements,

2. Identify the common tasks to be performed repetitively,

3. Determine a software flow diagram or class diagram,

4. Implement user interfaces and error handling, and

5. Debug, test, and verify the software.

All these steps should take place in parallel with documentation efforts. In my projects, I create what I call a “software design document” (SDD)--a live document that grows along with my software implementation. Most of the time, in industry, software maintenance and support continue years after the original software development and deployment. A well-written SDD becomes the most important living part at that time. An effective SDD should explain software functionality with respect to common modules and independent procedures, mapping, for example, global modules and variables and making them easily traceable by means of a global chart or traceability matrix.

Figure 1 (click for larger image) shows a graphical presentation of the predevelopment, development, and documentation steps for a typical software-development project.

In a typical software development cycle, a system designer and software developer cooperate in a predevelopment phase. Then, development and documentation occur in parallel.

Now, consider the five steps in detail:

1. Understand the requirements

Understanding software requirements is probably a software design team’s most important task. What should the software do? What are the software’s expected outcomes, and might various system limitations prevent you from achieving them? In reality, you’ll often face tradeoffs and need to make compromises in system specifications and software requirements. In such situations, the software developer and system designer need to work closely to generate a software-requirement document based on the customer specification.

For example, suppose a specification for a spurious-emission test requires measurement of spurs over a frequency range of 9 kHz to 40 GHz, using an Agilent Technologies of 8564X Series portable spectrum analyzer. An examination of the instrument’s data sheet will indicate that such a measurement won’t be possible with single sweep, because of dynamic-range limitations. As a result, the frequency range needs to be broken down into smaller segments—a fact that must be captured in the software-requirement document.

2. Identify common tasks

Identifying common tasks will facilitate the development of common modules to perform similar jobs more easily. Programming with all the code in a single file results in editing, debugging, and recompiling problems. In contrast, a program divided in to several modular source files reduces recompilation time, increases the efficiency of redoing a portion of the work, facilitates finding bugs in the shortest time, provides the option to reuse the smaller modules, and of course enables multiple personnel to work together in a team.   

To ensure the object orientation as well as proper functionality of common modules, a developer needs to specify the relationship of the modules within the software system. In simple words, a software developer will identify the inputs and outputs of common modules. Identifying the behavior of a common module requires extra effort to ensure the module’s integrity and flexibility.

For test-development purposes, common modules can handle tasks such as these:

c  communication to the unit under test (UUT),

c  instrument initializations, and

c  the repetitive setting of an instrument with specified parameters.

You can develop such modules in several ways: using a small LabView virtual instrument, for example, or using a class declaration in the Visual Basic or C programming languages. For an instrument-initialization module, you may need to identify the device address (or an array of addresses) as the input of the module, and, as the output of this common module, you may need to define a Boolean status (or an array of status values) that indicates successful communication with the instrument. Such a module or VI can be used by other tests for initialization. Since you can use these code blocks repetitively, you need to incorporate proper error handling—reusability’s benefits come with the burden of proving that your code is reliable.  

3. Software flow diagram

A software flow diagram is a graphical representation of your software-development strategy. This step in the development cycle becomes very important when multiple personnel are working on a project, with parallel development of software modules taking place. There are various ways of creating a flow diagram. Depending on program complexity, a development cycle may have multiple diagrams describing independent modular activity or data flow. A useful diagram should indicate the flow of the software in terms of the identified common modules, the graphical user interface, and the program inputs and outputs.

For example, in one of my transmit-test program-development cycles, I decided to develop four common modules:

c to set up an Agilent Model 4406 vector signal analyzer with specified settings,

c to start unit communication,

c to obtain pass or fail test limits from a configuration file, and

c to write to a results file.

Each of the tests required many extra steps—updating signal generators and reading temperatures, for instance—but all the tests used these common modules repetitively. For this application, I used a software flow diagram to capture the sequencing of all the tests using common modules along with the unique tasks.

4. User interface and error handling

4.1 User interface design.

For the end customer of the software, the user interface is as important as the understanding of the software requirements is to the software developer. A complex, unfriendly, highly-populated user interface will prevent the customer from enjoying the end product. When populating the user interface, the software developer must keep in mind that the user interface is the interface for users, not an interface for software developers or system designers. To meet your users needs, you may have to pay more attention to your software’s “look” and accessibility than to any other steps of your software-development project.

Let's discuss the look first. Sometimes, there is a common belief that a user interface for any kind of software should serve any users in general, including beginners and novices. I don't quite agree with that. A software developer needs to think about his or her specific customer—not the whole world. Intended audiences differ in many ways. For example, if you are designing database software for a company’s business group, you will think of one kind of user interface. If you’re developing the same software for an engineering group to handle test data, you might think of a completely different user interface. A customer-oriented user interface is the key here.

Keep in mind that simpler is always better. A simple user interface allows the customer to navigate easily and to understand the flow of the software. Introducing background pictures, cartoon characters, and flashing captions might seem to make a user interface more attractive initially, but this kind of user interface introduces complexity, is much more error-prone, and can annoy a daily user.

The second thing that a software developer needs to consider when designing a user interface is accessibility. This is very important to ensure proper functionality of the software. Let’s illustrate this point using an example. A few months back, I was working on a software module that would allow a user to plot and analyze data automatically. This Windows-based program was designed to provide the users with three available menu options: ‘Open Database File,’ ‘Create Analysis Criteria,’ and ‘Analyze Data.’ Now, if the user selects the menu option ‘Analyze Data’ prior to selecting the option ‘Open Database File,’ the program generates an error, because the program won’t know what file to analyze. You can be sure that some users some times will select inappropriate options if you as a software developer don’t carefully control the flow of the user interaction with the software. For this data-analysis program, I had to ensure that the menu options ‘Create Analysis Criteria’ and ‘Analyze Data’ were disabled until the user specified a file name.

Another very common example would be where a user needs to push a 'start' button to make something happen. If the 'start' button isn't disabled immediately after a user has pushed it, any event-driven software will have an error when user keeps pushing the start button. Even if you've taken care of the multiple events programmatically, it simply doesn't make sense to keep the 'start' button showing while something has already started.  Simply put, a programmer needs to populate the interface with objects on an “as needed basis.” As a developer, you should never expect the user to exercise perfect judgment.

4.2 Error/exception handling

Error-handling capacity often largely determines the reliability of any software. An exception can occur for various reasons: software bugs, hardware failure, and unforeseen circumstances. For most programming languages, if an error occurs, a programmer can direct the flow of the software to a defined state, provided the program checks for and detects the error. For example, if you are requesting a noise-figure reading from a noise figure meter with GPIB address of 2 using a GPIB address 3, an error will occur. Because of this error, the meter-response will be either some garbage value, or an empty value. Before moving to the next step in the program, a software designer needs to keep the option of error-checking for this response. This type of error checking is very common in test development. For programming languages like C, C++, or Visual Basic, the checking can be performed using a programmer-defined error function, which will check the value of the response in reference to the desired response. For a graphical test-development language like LabView, the error can be handled using a simple check on the error code of the language-defined error flow (error cluster).

Once a program detects an error in the normal flow of the execution, the action to be taken also needs to be determined. For this case, the program might simply exit from execution, notifying the user that the noise figure-meter with GPIB address 2 is not responding. A more sophisticated program may take an extra effort of making a device ID query to all the available devices in the system and change the GPIB address in the code to the proper one. During development, a software-sanity check can identify many problems, but planning ahead to handle exceptions due to unknown, unforeseen reasons might not be possible. The process of exception handling and the process of software verification therefore work in a closed-loop, iterative fashion.

In many companies, software engineers maintain a bug or exception traceability matrix to keep track of exceptions throughout the software lifecycle. A traceability matrix is used not only to keep track of the exceptions, but also to keep notes on the operational sequences and environments for which a failure occurs. For one of my test-development cycles, I had to measure power-detector accuracy for an ODU (outdoor unit) at room and extreme temperatures using LabView. After several days of successful tester operation, the test data started showing failures. I found out from the bug traceability matrix that for the first couple of days, the test was performed at room temperature, and that the failures occurred only at hot temperature. Knowing this helped me to dive immediately into my LabView sequence designated "Hot." As I mentioned earlier, it is not always possible to identify bugs from the code itself. Often, screen shots as well as the descriptions of failure situations can be the key factor for identifying the bugs.

5. Software reliability: debugging and testing

A system can be thought of as reliable when it behaves according to the specification or within expectation. Software reliability comes not only from the absence of bugs but also from the software’s capacity to handle abnormalities as well as environmental factors. For example, during one of my software-development cycles, I had to develop test software that would control a power meter-reading for an R&D unit with increasing input power. Under the test-development circumstances, the designated power meter was not responding to the GPIB commands reliably when I was sending too many commands sequentially. In a precise technical sense, this was not due to the software bug but due to the circumstances under which the software needed to operate. Nevertheless, successfully verified software means reliable and repeatable software. And software reliability also comes with the maturity of the code.

There are numerous industry standards available these days to verify the software reliability and repeatability. For R&D purposes, the most popular one is to calculate the statistical variations over a number of program executions. To measure repeatability, a software program is run usually five to ten times. The outcome of the program from one run is compared with that of the other ones. If the program is not designed to obtain any data, (i.e., a simple program designed to move files from a local directory to a server), then the software designer needs to verify whether the designated task has been accomplished successfully. However, if the program is designed to obtain raw data, (for example, 50 voltage readings from a power supply) then mathematically, you can parameterize this behavior using CpK (the process capability factor) calculations. Process-capability analysis is a measure of the data variations, from one set to another. The CpK factor also provides an indication of how reliable your program is. Simply put, data obtained from the five runs of the program under the exact same circumstances need to correlate with each other without much variation.

Before final software release, software developers need to verify their program using one or preferably all of the following three steps:

c Compare the outcome of the program with the desired expectation. For example, if, manually, a system designer has obtained a voltage reading of 12V, the automated software also needs to obtain the same value within tolerance limits.

c Use a preliminary release of the executable for beta testing to verify its reliability. The beta testing of a software program involves wide varieties of real people and their actual usage of the program. In beta testing, a number of different people will execute the program and will identify the issues, bugs, or expectations, which can be collected in a traceable matrix. Since the beta testing is supposed to occur almost at the end of the development cycle, many industries avoid performing extensive beta testing due to the shortage of funding as well as time. Nevertheless, in-house beta testing can also be used to ensure bug-free software.

c Perform a gage R&R study to determine the measurement variability as well as to determine what portion of the variability in measurements may be due to the measurement system and what portion of the variability is due to the software program. Usually, gage 5&5 (5 times software execution by 5 operators) serves as a standard for R&D sectors.

At the end of a successful development cycle, you should have a verified and reliable executable plus a number of documents: the requirement document, software design document, exception-handling document, verification document, and, of course, the software operating-procedure document. During my career, I have seen many software-development cycles end with an executable but with no documentation. Of course, in industry there is always a push to finish the project within its scheduled deadlines. However, maintaining at least the software design document, the bug traceability matrix, and software verification documents are the prime factors in reducing the cost of your software and ensuring its reliability throughout its life cycle.

© 2003 All Rights Reserved. Taniza Afrin. E-mail: tafrin@hns.com.