Smarter testing tames the explosion of embedded software

Chris Washington- November 27, 2012

Consumer demand for new features and increasing regulations from governments and related organizations is fueling an explosion of embedded software in mechanical systems. Using a network of sensors and actuators, embedded software can sense and respond to user commands and external disturbances improving the performance and/or functionality of mechanical systems.

The most common example is the automobile which has grown from no embedded software to as many as 10M lines of code in a single vehicle in less than 30 years and is expected to surpass 100M lines of code within the next 2 decades. But this isn’t the only industry embracing the use of embedded intelligence. Aerospace, medical devices, energy harvesting and power generation have all been using embedded software in their systems for many years.  Even white goods are well on their way down this path with smart washing machines and dryers that can contain as much as 100k lines of code.
 
This is great news for the continued advancement of these products, but for test engineers it presents a formidable challenge. The growing role of embedded software in these systems produces a proportional growth in potential errors affecting the quality of the system. NASA commissioned a study to see how few errors could be achieved in a complex embedded program and using all the resources and expertise they are known for were only able to achieve between 0.1 and 1.0 software errors (“bugs”) per 1000 lines of code. Unfortunately, the timelines and budgets allocated to find these errors are not and cannot grow proportionally. Given these two challenges it is clear that test engineers must find a way to test earlier, test more efficiently and test more thoroughly.

To accomplish these goals, engineers have expanded their testing tool belt to include smarter test systems that leverage the simulation models used in the design of the system. One example is hardware-in-the-loop (HIL) testing. HIL testing is used to test the software deployed on the production electronic control unit (ECU) without using the physical system it will be controlling. The test systems use models of the system and the environment it will operate in to simulate the electrical interaction between virtual system (or more specifically, its virtual sensors and actuators) and the actual ECU being tested.



Because these plant models are typically used by design engineers to develop the embedded software, they have sufficient fidelity for testing activities allowing the engineer to being the verification of the deployed embedded software before the mechanical system is even available.  Also, because there are no “moving-parts” in this form of testing – only electrical signals from simulated sensors and actuators – engineers can use test automation to queue up a series of test scenarios to execute over night or during the weekend increasing test coverage. And finally, this approach allows test engineers to safely and repeatably create undesirable scenarios to verify the software’s ability to identify and handle faults in system operation.

While the use of HIL testing provides many benefits to the validation of embedded software systems, it does not eliminate the need to physically test the system. Instead, it allows more errors to be discovered and resolved prior to physical testing - minimizing the number of the more expense, less flexible testing tasks necessary to validate the system. As system designers continue to expand the role of software in mechanical systems, the efficient use of this and other smart testing techniques will be critical to their success by enabling projects to stay on-time and on-budget without sacrificing quality.

About the Author
Chris Washington is the segment manager for Embedded Software Validation and Real-Time Testing. During his tenure at NI, Washington has served as the product manager for NI VeriStand, LabVIEW Control Design and Simulation Module, LabVIEW State Chart Module, and LabVIEW for embedded applications. Prior to these roles, he spent three years in Detroit, MI, as a field engineer where he provided consulting and support services for various applications, such as hardware-in-the-loop testing; rapid control prototyping; noise-vibration-harshness testing; in-vehicle data logging; and real-time test cell development. Additionally, Washington has worked as an applications engineer and a LabVIEW instructor. He received his bachelor of science in electrical engineering from Texas A&M University with a focus on digital electronics and control systems.