Embedded design survey reveal safety concerns

Barr Group's annual release of Embedded Systems Safety & Security Survey puts into focus the importance of safely using the best practices and safety standards in the design of the device. Data analysis from over 2,400 responses, all from engineers working on embedded systems design, were just gathered.

With such a large sample of engineers from all over the world (46 per cent from North America and 33 per cent from Europe), we were excited to learn more about the design philosophies and practices of engineers, as it relates to safety and security. We shared many of the results last week at Embedded World in Germany, and we will be sharing more results at our upcoming free webinar on March 8, providing more details (such as geographic preferences and distributions by experience level).

But, there are some troubling trends that should not wait—and that everyone should stop and ponder now.

Would it surprise you to know that 22 per cent of our respondents are currently working on device designs that can kill? We asked what was the worst thing that could happen if the device you are designing today were to malfunction in the field and more than 500 respondents said one or more people could die! Many of these respondents are in the industrial automation, medical device, automotive, and aerospace/defence industries.

It's not unexpected that these industries create devices that are safety-critical, but with such a large response, we wanted to know whether these designers were following safety standards and following best practices for reliability and maintainability. IEC, FDA, FAA, NHTSA, SAE, IEEE, MISRA, and other professional agencies and societies work to create safety standards for engineering design. With these standards in place, my hope was that the affirmative responses would be close to 100 per cent.

Unfortunately, that was not the case.

Only 67 per cent are designing to relevant safety standards, while 22 per cent stated that they are not—and 11 per cent did not even know if they were designing to a standard or not. Let's contemplate that for a moment. If we take this at face value, that means that approximately one out of every three safety-critical device designs has potential safety, reliability, security, or quality holes that are not being adequately addressed or vetted. This is quite disturbing.

Let's go a little deeper on this. Industry safety-standards compliance can be costly and time consuming, but what about other best practices that are part of good design, such as use of coding standards, code reviews, and static analysis? The news here is disconcerting too. For this group of engineers designing devices that can kill, the following graphic tells a compelling story:

Figure 1: Coding practices among developers working on products with a deadly failure mode.

Why are these numbers not near 100 per cent utilisation?

As we presented these results at Embedded World, I saw many reactions—primarily surprise and scepticism. There was concern that these results could be true and questions about whether our numbers were wrong.

The sceptics had many comments. Some wondered whether our numbers were skewed because not all of our respondents were software engineers and might not know the status of best practices on software development. We do not think that is an issue because of the demographics of the data (including the fact that just 6 per cent of respondents were involved only in hardware).

Some also wondered if there was geographic skew to the data, but again, because we had a broad response from North America, Europe, and Asia, we believe the numbers are a good approximation of engineers' thoughts.

Some wondered if designers of non-safety-critical sub-systems within a safety-critical device (e.g., the satellite radio within an automobile) might be affecting the results. But, as we have seen, with today's interconnected devices and security challenges, even non-critical sub-systems can affect other sub-systems.

Our results urge us all to address this situation. Managers need to understand the importance of safety and security and that it needs to be baked into project schedules and budgets.

The fact is that we all need to acknowledge in this age of IoT that our devices are becoming more critical to the infrastructure of the world. We all must devote the time, resources, and dollars to improving reliability. If we do so, in the long run, lives will be saved (and, for those business managers out there, money will be saved too).

- Andrew Girson