[trustable-software] Implementing Trustable Software

trustable at panic.fluff.org trustable at panic.fluff.org
Wed Feb 22 00:57:28 UTC 2017


At the heart of the tooling issues for Trustable Software are a series of 
desires.

    * Auditablity of the actions of the system.
    * Reduction of human effort
    * Parralisation to shortend cycle time.
    * Consistent application of policy


         Driving the auditability of the systems is the needs to have the 
changes and the metadata about the changes under the same access control 
mechanisms and stored in a common location to ensure that it is impossible 
to subvert one of the data or metadata, throwing things into uncertainty.

 	Driving the reduction of human effort is automation. In order to 
support flexible changing workflows of the modern workplace, it is 
essential to build systems from composable, reusable blocks which allow 
one to model the necessary workflows through adjustment of policy.

 	Driving parallelisation is concepts of event driven pipelines and 
bounded development.

 	Driving the consistent application of policy is how we ensure that 
we deal with implementation of requirements and the merge of changes.
There are long discussions about this in various places but this is a good 
overview
<https://barro.github.io/2016/02/a-succesful-git-branching-model-considered-harmful/>

What is missing and so needs developing ?
=========================================

     1] A metadata library for the source code control system.

       Recent development within the git community have been driven around 
the concept of NoteDB
       <https://storage.googleapis.com/gerrit-talks/summit/2015/NoteDB.pdf>
and this is now being used by a number of review systems. The development 
of a reference library with language bindings to a number of common 
languages would allow more common use of NoteDB.

       It is my view that language used to implement the library is 
un-important but their should be language bindings certainly for Perl (as 
this is used widely within the Git community and Python (as this is used 
widely for configuration tools in Linux)

       The current convention is to place machine parsable data in commit 
messages to provide linkage to requirement or issue tracking systems. 
This is very common in the git community, often descrived as git-footer 
handles.

      There is an argument that this should be handled by a seperate 
library as it differnt git data.

      It is my view that we need a library to handle these git-footers, 
with language bindins at least to the same ones as the NoteDB library.

      2] Gating Tool

      Something capable of evaluation which workflows to execute based on 
the contents of repositories, the associated metadata and test data.
There are number of these available today, however they are often 
conflated with other tasks such as job scheduling or reviewing.

        It is my view that we require a simple tool which can be event 
driven and having evaluated these gate can cause the creation of jobs 
which result in the update the repositories and associated metadata 
which may in turn cause further evaluation.


      3] Job Scheduler

      We want to distrubute work based on available resources and have them 
operate asynchronous to maximise parralism within the system and ideally 
minimuise the cycle time to confirm a functioning product.

      There is a long history of systems which are capable of this, from 
the batch systems such as HTCondor through to OpenStack's Nodepool to 
Hasicorp's Nomad project. I have a particular fondness for DAGMan part of 
HTCondor.<https://research.cs.wisc.edu/htcondor/dagman/dagman.html>

      It is my view that we should provide a library which could abstract 
these services to a common interface.


What have I missed ?
====================

       So... what have I missed ? Is this an overly simplistic approach

Edmund


-- 
========================================================================
Edmund J. Sutcliffe                     Thoughtful Solutions; Creatively
<edmunds at panic.fluff.org>               Implemented and Communicated
<http://panic.fluff.org>                +44 (0) 7976 938841




More information about the trustable-software mailing list