[trustable-software] Exploring the "Hypothesis for software to be trustable"

Paul Sherwood paul.sherwood at codethink.co.uk
Tue Jan 2 10:41:00 GMT 2018


Hi folks,
over the break I was thinking about the hypothesis that Edmund expressed 
last year [1], and found myself attempting to codify some of the context 
and underlying assumptions, as you can see below.

I've gone for a shorthand of t.* to denote things that seem to be either 
required or implied by the 'Trustable Software' discussions and 
concepts.

Note that:
- I'm not even confident yet that the hypothesis is generally correct.
- I think that the specifics of the hypothesis are probably not all 
correct
- I'm completely sure that following statements contain bugs and I would 
appreciate any/all review feedback

br
Paul

# t.software

- we assess the trustability of t.software based on t.evidence
- t.software MUST have t.requirements
- t.software MUST satisfy its t.requirements
- there MUST be t.measurable t.evidence that t.software satisfies its 
t.requirements

# t.requirements (derived from wikipedia [2])

   - t.requirements MUST be unitary
   - t.requirements MUST be atomic (non-conjugated)
   - t.requirements MUST be complete
   - t.requirements MUST be consistent
   - t.requirements MUST be traceable
   - t.requirements MUST be current
   - t.requirements MUST be unambiguous
   - t.requirements MUST be verifiable
   - t.requirements MAY be graded by priority/importance

# t.tests

- t.requirements MUST exist
- all t.requirements MUST have t.tests
- all t.tests MUST pass
- the scope of the t.requirements MUST be t.measurable
- the scope of the t.tests MUST be t.measurable
- we MAY t.estimate the scale or scope of missing t.requirements
- we MAY t.estimate the scale or scope of missing t.tests

# t.evidence

- t.evidence is machine-parseable metadata associated with t.software
- t.evidence MUST include t.evidence of its own integrity/correctness
- t.evidence MUST include t.identity for author(s)
- t.evidence MUST include t.identity for committer/uploader
- t.evidence MUST include creation/modification/commit time-and-date
- t.evidence MAY include text description and/or comments
- t.evidence MAY include t.identity for reviewer(s)
- t.evidence MAY include t.identity for review comments


# t.identity, t.identify

- t.identity is a kind of t.evidence
- t.identity MUST map to a single t.contributor
- a t.contributor is a uniquely identifiable person, organisation or 
software program
- t.contributors contribute to t.software and/or t.evidence
- t.contributors are responsible and accountable for their contributions

# t.measure, t.measureable, t.estimate, t.measurement, t.metrics

- t.measurements MUST be derived from t.evidence via automation
- t.estimates MUST be derived from t.evidence via automation
- t.measurements MUST be believed to be factually correct
- t.estimates are known to be uncertain
- we t.measure all available t.evidence about t.software, producing 
t.metrics
- we t.measure multiple t.metrics, including signals for some (ideally 
all) of
   - engineering cost/effort
   - project scale
   - project value
   - project quality
   - project software performance/efficiency
- we use t.estimates to check t.measurements or fill in gaps in 
t.evidence
- we look conflicts between t.metrics
   - we evaluate the conflicts
- we aim to identify gaps in t.metrics
   - we seek new t.evidence to close them
- all t.metrics MUST be meaningful
   - for t.software, all t.metrics are acceptable
- we can aggregate/manipulate t.metrics to provide a t.measure of 
trustability

# t.provenance

- t.evidence MUST t.identify the authors of the t.software
- t.evidence MUST t.identitify the reviewers of the t.software
- t.evidence MUST t.identify the integrators/mergers of the t.software
- FIXME add hypothesis provenance logic here

# t.change

- a t.change is is any change to standards, requirements, tests, code,
   documentation, or other materials that affect the project
- a t.change specifically includes the initial upload of any component 
of any
   materials

# t.build

- a t.build consists of any compilation, reformation, rearrangement,
   or copying of any sources to generate any artefacts that may become
   subject to test or deployment
- FIXME add hypothesis build logic here

# t.process, t.effect, t.friction, t.value

- t.process is a set of practices to increase the trustability of 
software
- adding t.process to a project MUST lead to t.measurable t.effects
- t.effects are t.measures and/or t.metrics
- t.friction is negative t.effects
- t.value is positive t.effects
- the t.effects of adding t.process to any project MUST be t.measurable
- adding t.process to a mature infrastructure project must
   - provide t.measurable t.value
   - involve acceptable (minimal) t.friction
   - be justifiable to be used upstream and/or
   - be applicable downstream

# t.reproduce

- FIXME add hypothesis reproduction logic here

# t.function

- FIXME add hypothesis functionality logic here

# t.update

- FIXME add hypothesis update logic here

# t.context

- t.context is written information which states or clarifies relevant 
ideas for t.software
- [trustable software 
discussions](https://lists.trustable.io/pipermail/trustable-software/)
- [wikipedia requirement](https://en.wikipedia.org/wiki/Requirement)
- [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt)
- virtually all existing software does not satisfy the [trustable 
software 
hypothesis](https://gitlab.com/trustable/overview/wikis/hypothesis-for-software-to-be-trustable)
   - but a lot of software is widely trusted
- to be useful, the t.software concepts must be
   - easy to understand
   - justifiable
   - practical and realistic
   - applicable to existing software
     - collecting t.evidence needs to be low friction
     - t.measures and t.metrics for widely trusted software existing 
software  should be good


[1] 
https://gitlab.com/trustable/overview/wikis/hypothesis-for-software-to-be-trustable
[2] https://en.wikipedia.org/wiki/Requirement




More information about the trustable-software mailing list