[trustable-software] RFC: Software to be Trustable
trustable at panic.fluff.org
trustable at panic.fluff.org
Tue Jan 31 10:34:19 UTC 2017
After some long discussions with Emmet Hikory
<emmet.hikory at codethink.co.uk> and Paul Sherwood
<paul.sherwood at codethink.co.uk> I'd like to offer the following as a
defintions for Software to be Trustable.
A "change" is any change to standards, requirements, tests, code,
documentation, or other materials that affect the project, specifically
including the initial upload of any component of any materials.
A "build" consists of any compilation, reformation, rearrangement,
or copying of any sources to generate any artefacts that may become
subject to test or deployment.
We know where it comes from:
Any upload of any change must be certain to belong to some uploader.
This implies an authentication mechanism in the patch tracker.
This implies a mechanism to override SCM provenance information
(Alice may assert Bob wrote a patch, but it was uploaded to
the tracker by Alice, so the system must know this).
Any upload of any change must have a facility for attestation of origin
This implies metadata about changes beyond commit messages
No import of foreign sources is without attestation of origin
The SCM must have a gated branch with continuous history
We know how to build it:
Any build must be performed by automation.
This implies the build instructions must be machine readable
This implies the build environment is produced by automation.
We have a complete log, including environment details, of at least
two apparently identical successful builds.
We can reproduce it:
Any build performed twice must generate the same results.
This implies the sources must allow reproducible build
This implies the build environment must be isolated
Any deployment performed must be performed by automation
This implies the deployment instructions must be machine readable
This implies that some trustable software does not have
identical state after deployment.
It does what it is supposed to do:
It does not do what it is not supposed to do:
Any requirement must be expressed in a manner that can be verified
This implies the requirements are machine readable
This implies tooling to generate tests based on the requirements
The result of any test is machine readable
Any software deployed to production has prior evidence of passing tests
This implies pre-merge functionality in CI or similar
Any change must be associated with a requirement
There must exist information indicating the minimum tests required to gate
These tests must encompass all requirements related to the subject
of the change
We can update it and be confident it will not break or regress:
Any update must undergo functional testing
This implies pre-merge integration and testing.
This implies stable build and deployment facilities
If all these statements are true then I would postulate that the software
could be regarded as Trustable..
I look forward to your opinions and views
Edmund J. Sutcliffe Thoughtful Solutions; Creatively
<edmunds at panic.fluff.org> Implemented and Communicated
<http://panic.fluff.org> +44 (0) 7976 938841
More information about the trustable-software