[trustable-software] Implementing Trustable Software
trustable at panic.fluff.org
trustable at panic.fluff.org
Wed Feb 22 00:57:28 UTC 2017
At the heart of the tooling issues for Trustable Software are a series of
desires.
* Auditablity of the actions of the system.
* Reduction of human effort
* Parralisation to shortend cycle time.
* Consistent application of policy
Driving the auditability of the systems is the needs to have the
changes and the metadata about the changes under the same access control
mechanisms and stored in a common location to ensure that it is impossible
to subvert one of the data or metadata, throwing things into uncertainty.
Driving the reduction of human effort is automation. In order to
support flexible changing workflows of the modern workplace, it is
essential to build systems from composable, reusable blocks which allow
one to model the necessary workflows through adjustment of policy.
Driving parallelisation is concepts of event driven pipelines and
bounded development.
Driving the consistent application of policy is how we ensure that
we deal with implementation of requirements and the merge of changes.
There are long discussions about this in various places but this is a good
overview
<https://barro.github.io/2016/02/a-succesful-git-branching-model-considered-harmful/>
What is missing and so needs developing ?
=========================================
1] A metadata library for the source code control system.
Recent development within the git community have been driven around
the concept of NoteDB
<https://storage.googleapis.com/gerrit-talks/summit/2015/NoteDB.pdf>
and this is now being used by a number of review systems. The development
of a reference library with language bindings to a number of common
languages would allow more common use of NoteDB.
It is my view that language used to implement the library is
un-important but their should be language bindings certainly for Perl (as
this is used widely within the Git community and Python (as this is used
widely for configuration tools in Linux)
The current convention is to place machine parsable data in commit
messages to provide linkage to requirement or issue tracking systems.
This is very common in the git community, often descrived as git-footer
handles.
There is an argument that this should be handled by a seperate
library as it differnt git data.
It is my view that we need a library to handle these git-footers,
with language bindins at least to the same ones as the NoteDB library.
2] Gating Tool
Something capable of evaluation which workflows to execute based on
the contents of repositories, the associated metadata and test data.
There are number of these available today, however they are often
conflated with other tasks such as job scheduling or reviewing.
It is my view that we require a simple tool which can be event
driven and having evaluated these gate can cause the creation of jobs
which result in the update the repositories and associated metadata
which may in turn cause further evaluation.
3] Job Scheduler
We want to distrubute work based on available resources and have them
operate asynchronous to maximise parralism within the system and ideally
minimuise the cycle time to confirm a functioning product.
There is a long history of systems which are capable of this, from
the batch systems such as HTCondor through to OpenStack's Nodepool to
Hasicorp's Nomad project. I have a particular fondness for DAGMan part of
HTCondor.<https://research.cs.wisc.edu/htcondor/dagman/dagman.html>
It is my view that we should provide a library which could abstract
these services to a common interface.
What have I missed ?
====================
So... what have I missed ? Is this an overly simplistic approach
Edmund
--
========================================================================
Edmund J. Sutcliffe Thoughtful Solutions; Creatively
<edmunds at panic.fluff.org> Implemented and Communicated
<http://panic.fluff.org> +44 (0) 7976 938841
More information about the trustable-software
mailing list