[trustable-software] RFC: Software to be Trustable

trustable at panic.fluff.org trustable at panic.fluff.org
Tue Jan 31 10:34:19 UTC 2017


After some long discussions with Emmet Hikory 
<emmet.hikory at codethink.co.uk> and Paul Sherwood 
<paul.sherwood at codethink.co.uk> I'd like to offer the following as a
defintions for Software to be Trustable.


Nomenclature:

     A "change" is any change to standards, requirements, tests, code,
documentation, or other materials that affect the project, specifically
including the initial upload of any component of any materials.

     A "build" consists of any compilation, reformation, rearrangement,
or copying of any sources to generate any artefacts that may become
subject to test or deployment.



We know where it comes from:
     Any upload of any change must be certain to belong to some uploader.
         This implies an authentication mechanism in the patch tracker.
         This implies a mechanism to override SCM provenance information
             (Alice may assert Bob wrote a patch, but it was uploaded to
              the tracker by Alice, so the system must know this).
     Any upload of any change must have a facility for attestation of origin
         This implies metadata about changes beyond commit messages
     No import of foreign sources is without attestation of origin
     The SCM must have a gated branch with continuous history

We know how to build it:
     Any build must be performed by automation.
         This implies the build instructions must be machine readable
         This implies the build environment is produced by automation.
     We have a complete log, including environment details, of at least
       two apparently identical successful builds.

We can reproduce it:
     Any build performed twice must generate the same results.
         This implies the sources must allow reproducible build
         This implies the build environment must be isolated
     Any deployment performed must be performed by automation
         This implies the deployment instructions must be machine readable
         This implies that some trustable software does not have
             identical state after deployment.

It does what it is supposed to do:
It does not do what it is not supposed to do:
     Any requirement must be expressed in a manner that can be verified
         This implies the requirements are machine readable
         This implies tooling to generate tests based on the requirements
         The result of any test is machine readable
     Any software deployed to production has prior evidence of passing tests
         This implies pre-merge functionality in CI or similar
     Any change must be associated with a requirement
     There must exist information indicating the minimum tests required to gate
        These tests must encompass all requirements related to the subject
        of the change


We can update it and be confident it will not break or regress:
     Any update must undergo functional testing
         This implies pre-merge integration and testing.
         This implies stable build and deployment facilities


If all these statements are true then I would postulate that the software 
could be regarded as Trustable..

I look forward to your opinions and views
Edmund


-- 
========================================================================
Edmund J. Sutcliffe                     Thoughtful Solutions; Creatively
<edmunds at panic.fluff.org>               Implemented and Communicated
<http://panic.fluff.org>                +44 (0) 7976 938841




More information about the trustable-software mailing list