[trustable-software] Requirements and architecture for Safety

[Paul Sherwood]

Over recent months I've been attempting to understand the implications 
of safety and security, which have always seemed to me to be the most 
difficult of the seven factors we've identified as material for 
trustable software.

In this email I'll summarise my current understanding for Safety

- safety is a system property - we should only consider safety in the 
context of a whole system
- the most widely used approaches to safety focus on reliability, and 
this leads to specific demands for reliable software
- safety standards make demands about the processes to be applied for 
critical software

If we are aiming for trustable software in a safety-critical system, we 
need

- evidence that risks and hazards inherent in the system and its 
environment which could cause harm have been identified
- evidence to show how the system has been designed/architected to deal 
with the risks and hazards
- evidence to show that the software behaves as expected to support the 
safety design/architecture

I've made myself quite unpopular on the System Safety Mailing List [1] 
in pointing out that lots of successful software has been and will 
continue to be created without traditional 'requirements' or 
'architecture'. So far I don't see that the lack of such documents for a 
piece of software is a red flag for consuming it in a safety critical 
system.

However I think I'm now clear that we can only make claims about safety 
of a system if we can provide evidence to show how we believe we have 
made the system safe.

So it seems to me that we can't close this gap for trustable software, 
without evidence that we have

a) captured our system-level safety requirements, based on prevention of 
losses due to identified risks and hazards
b) described our safety architecture to address the safety requirements
c) specified the required properties/behaviours of components within the 
architecture (e.g. the software that we hope to be trustable)
d) tested that the properties/behaviours are delivered

All of this evidence should be expected to evolve, so as usual I'd 
expect the documentation to be kept under version control, with history 
and provenance, and actively maintained in CI/CD.

I've not thought so much about the Security topic yet, but my current 
instinct is that similar reasoning must apply, ie we need system 
security requirements, a system security architecture, and evidence that 
chosen components deliver desired behaviours to support the 
architecture.

br
Paul

[1] http://systemsafetylist.org