Embedded software applications challenge test limits
Technical evangelist Bill St. Clair discusses the development of embedded software for avionics and automotive applications.
Greg Reed, Contributing Editor -- Test & Measurement World, 5/9/2008 2:32:00 PM
Bill St.Clair, who has more than 25 years of experience developing embedded software, has worked in the avionics, defense, space, communications, and industrial control industries. He has worked as a developer, verification engineer, and manager on projects covering all phases of the software life cycle. St. Clair has been employed by General Electric, Unisys, and IBM-Rational, and he is currently technical evangelist for LDRA Technology, a company that provides automated analysis and testing tools for mission-critical software applications.In this exclusive interview, St. Clair discussed the development of embedded software for avionics and automotive applications.
Q: What are some special considerations for developing test software products for avionics, defense, and automotive applications?
A: When developing software, companies need to easily see that the source code they are producing is secure and fault-free and adheres to the required quality standards. Managers and team workers, as well as individual developers, also need the abiltiy to monitor testing and quality metrics.
Compliance with certification standards is important whether it be CMMI, DO-178B for avionics, EN-50128 for railway transportation, or IEC 61508 for automotive. When developing software, companies also have requirements specific to the standard they want to meet, so this is where software tools can assist with automating some of these processes.
Q: What test software advancements in aerospace may be applied to the automotive sector?
A: Requirements-based verification is becoming increasingly important, as this technology ties all the aspects of software verification and test into an automated process. Through requirements traceability, analysts and project managers can see a complete picture of traceability from software requirements to design, code, test specifications, and test results.
Advanced areas include the ability for users to be able to create and link software verification plans and reports between requirements and all verification activities. This leads to the ability to be able to scale requirements management for projects of all sizes and foster more of a systems engineering approach to software verification and test.
The flow-down of requirements from multiple sources including requirement management tools such as Telelogic DOORS, IBM Rational Requisite Pro, Microsoft Word documents, and spreadsheets via a bidirectional link is a concept that brings together technologies that previously had been manual, time-consuming processes focused on requirements traceability and verification. Prior to automation, these manual activities represented 50 to 70% of project costs and were absolutely critical to successful software development.
Q: Please elaborate on the role programming standards play in the aero/auto sectors.
A: Before the advent of high-level programming languages, everything was handcrafted in machine code, using slow, torturous processes. Because of this, great care was taken to ensure the code was correct, as corrections took a long time to implement.
The introduction of higher-level programming languages and better program-entry methods ended this tedium, at least in theory. In reality, people started to find that they had programs that did not work as expected, and effort simply moved from coding to fault-finding.
Numerous organizations and authors analyzed the defects that appeared in a wide set of programs and realized that many had common failure causes. This information enabled the production of guidelines, the aim of which was to prevent, or make detectable, the issues that could lead to the introduction of defects.
Q: What impact do you expect from the adoption of MISRA C and the soon to be released MISRA C++ ?
A: MISRA is a collaboration between automotive manufacturers, component suppliers, and engineering consultancies that seeks to promote best practice and commonality in the development of safety-related automotive electronic and other embedded systems through the publication of standard guidelines. Since its launch, the success of MISRA-C as a “best practice” solution has not only seen its application spread throughout the worldwide automotive industry, but it has also been increasingly adopted for safety-related and safety-critical software-development projects and applications in a wide variety of other industries including aerospace, military, rail, and medical sectors.
With the success of the MISRA-C standard, a MISRA-C++ committee was established in 2005 to work toward the creation of standard MISRA guidelines for the C++ programming language. The standard, due to be released soon, will have as much of an impact on the industry as its successful predecessor MISRA C.
Q: How do information metrics and reporting capabilities become a vital part of software defect control?
A: Because embedded software is becoming increasingly complex, best programming practices must be increasingly applied to embedded software development. It is likewise increasingly important that these best practices enable processes facilitating zero-defect software development with an emphasis on the causality of defects instead of their effects. Such a focus is mandated by CMMI Level 5.
Best practices begin with requirements validation and the determination of requirements testability. With this approach, a requirements traceability matrix must be maintained throughout the software-development life cycle. Customers frequently relate that 70% or more of defects are requirements related. Best practices also include the application and reporting of rigorous and comprehensive coding rules to source code. These rules preclude the vast majority of run-time defects. As test cases are executed, the code coverage must be measured, verifying the correct functionality and the robustness of software.
Q: How do non-avionics based suppliers adopt the DO-178B standard and grapple with object code verification?
A: An ever-increasing reliance on software control has meant that many companies from the automotive and other business sectors that do not have a traditional requirement for sophisticated software analysis now find themselves compelled to undertake safety-critical or safety-related testing.
With this increased requirement for software testing across different industries, a tendency has emerged for companies to look outside their own market sector when seeking best practice techniques or standards. Examples of such industry crossovers have been seen in the automotive and avionics industries with the adoption of elements of the DO-178B standard in the former and a similar adoption of the MISRA standard in the latter.
With out-of-sector testing standards comes the potential for unfamiliar testing techniques. This is illustrated by the object-code verification requirements of the DO-178B standard. While a key testing element of many avionics programs, it has been a relatively unused technique outside avionics. The increasing sophistication and safety-critical nature of many modern embedded control applications, however, mean that as non-avionics based suppliers adopt DO-178B, they must pay attention to object-code verification.
You might ask, what is object-code verification? In a nutshell, object-code verification is concerned with how much the control flow structure of the compiler-generated object code differs from that of the application source code from which it was derived. Such differences may occur for a number of reasons, such as compiler interpretation or optimization. Given, however, that traditional structural coverage techniques are applied at the source code level even though it is actually the object code that executes on the processor, differences in control flow structure between the two can make for significant gaps in the testing process.
The demands of DO-178B are such that developers of applications that are subject to the standard are required to implement object code verification facilities for those elements of the application that have a Level-A (safety-critical) classification.
Q: What metrics do you advise customers to use when purchasing test software and equipment?
A: Customers need to ask, “Does the tool measure the artifacts produced as a result of best practices? Does the tool seamlessly integrate with the correct processes?” If the metrics simply reflect the occurrences of run-time defects after software integration has occurred while concealing the root causes of these defects, then the tool measures too little too late. Customers need to ensure that the tool actually encourages and enables software development best practices such as requirements-based development and verification if they want to make sure that they are truly adopting a software-quality-management approach.
Q: What automation advancements do you expect to become incorporated into test equipment over the next three to five years?
A: Test specifications will automatically be derived from requirements and software artifacts to generate a highly robust suite of functional tests. Tools that attempt to circumvent best practices such as run-time error tools and formal methods development tools that preclude testing will be displaced.
Q: Any final comments?
A: Safety- and security-critical software systems dictate conformance to best practices. Most other new development such as mission- and business-critical systems will increasingly see the (cost) benefits of best practices and will look to tools that reinforce these best practices.
Talkback
Related Content
Sponsored Links
TMW Resource Center
Browse Resources by Type:
- Access to EASE (Equipment Appraisal Search Engine)
DispoDog | Product Demo
VIEW NOW
Agilent | White Paper
VIEW NOW
View More
