[C-safe-secure-studygroup] Testing

Clive Pygott clivepygott at gmail.com
Thu Jan 5 13:10:07 UTC 2017


I'm with Andrew on this - testing has to be out-of-scope

Testing is a hugely complex and contentious topic. Telling an organisation
like RTCA how they should be doing avionic certification is not going to go
down well - its only recently that there's been a five year battle to get
formal methods accepted as an alternative approach to MCDC and other
coverage based techniques. Our saying 'this is how software should be
tested', without any (I'm assuming) domain experience is not going to be
credible

Whilst I'd also agree that we have to be C focused - we are after all a
study group of the C language committee - I'd suggest that there is scope
for some creativity, for example in proposing attributes to assist static
analysis or to clarify programmer intention (e.g. most safety standards ban
fall-through from one case clause to another, because there's no way to be
sure whether this was intentional or not, even though there's nothing
inherently risky in doing it. If a 'fall-through' attribute can be added,
this facility can be used, with the programmer asserting that they are
aware of what they are doing)

     Clive

PS:  One thing this discussion on testing does indicate is that there is a
need for a clear scoping statement for this study group.  Maybe that should
be an objective for Markham?
.

On Thu, Jan 5, 2017 at 7:10 AM, Andrew Banks via C-safe-secure-studygroup <
c-safe-secure-studygroup at lists.trustable.io> wrote:

> On Wed, Jan 4, 2017 at 3:56 PM, Wheeler, David A via
> C-safe-secure-studygroup <c-safe-secure-studygroup at lists.trustable.io>
> wrote:
>
> > Dynamic analysis, including fuzzing and traditional testing, requires
> > that you be able to execute the code.  There are many circumstances
> > where you have the source code but cannot run it directly (e.g.,
> > because you need special hardware).  In addition, many tool suppliers
> > do only static analysis, or only dynamic analysis – not both.
> >
> > I think it does **NOT** make sense to “bundle” these two different
> > activities in a single spec.
> >
> > I recommend that dynamic issues, like fuzzing and testing, be handled
> > **separately** by a different spec or different group.
>
> Agreed... this activity needs to focus on addressing issues with the C
> language
>
> Testing (including methods such as fuzzing) should be outside the scope...
> in fact there is already an ISO WG looking at Software Testing (ISO 29119)
>
> As a further distinction, I suggest this group should avoid becoming
> embroiled with generic (ie non C-specific) static code analysis level
> activity - unless that is planned as a separate document!
>
> Andrew
>
>
> _______________________________________________
> C-safe-secure-studygroup mailing list
> C-safe-secure-studygroup at lists.trustable.io
> https://lists.trustable.io/cgi-bin/mailman/listinfo/c-
> safe-secure-studygroup
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170105/a236cca7/attachment.html>


More information about the C-safe-secure-studygroup mailing list