[trustable-software] A breakthrough (in my mind, at least)

[Paul Sherwood]

Hi all

Over recent weeks I've been struggling to understand how various 
communities deal with safety (in general, not just software).

Until last week I was finding occasional moments of insight (e.g. "Aha, 
the IEC standards business model is a disgrace"), but mainly I was 
getting drawn into the rabbit holes of 'safety argumentation', 'safety 
manual', 'pre-certification', ''probabilistic risk assessment" etc.

This has been making me increasingly uncomfortable, since I personally 
don't believe that the defacto approaches (including, sadly, the 
MISRA/CERT C standard work, hence my cross-post to the study group list) 
are widely fit-for-purpose in the world in which I find myself. We're 
designing extremely complex systems, containing huge amounts of 
software. Most of that software is and will continue to be produced by 
people who will never follow the MISRA/CERT standards.

And they certainly won't read ISO26262 or IEC 61508 (who in their right 
mind would actually pay $3000 for that???)

The elephant in the room for me is autonomous vehicles - no-one can 
credibly claim that all of that software is going to behave 
deterministically, let alone be safe and secure (in the real human 
sense) if our best defence is what was mapped out by 
embedded/electronics engineers decades ago.

So in my heart of hearts I was beginning to think "eek... there's 
**way** more work to do than i ever thought possible".

And then, bingo. I came across Nancy Leveson. The work has been done! 
Crisis averted! Nancy and her colleagues at MIT have been working for 
decades on a systematic, defensible, grown up engineering approach to 
safety [1]

And security!

And other emergent properties of complex systems.

Happily, this work is freely available. So I don't have to pay for unfit 
documents that will stay unfit because hardly anybody actually reads 
them, and it's too hard to convince the rights holders that their babies 
are ugly.

So now the fun starts. I've asked colleagues to start working on FTPA 
hazard analysis for autonomous vehicles. IIUC understand it, it won't 
take very long :-)

If you'd like to contribute please let me know.

br
Paul

[1] http://psas.scripts.mit.edu/home/