[C-safe-secure-studygroup] Static vs dynamic analysis: who cares?

Robert Seacord rcseacord at gmail.com
Fri Jan 6 17:18:06 UTC 2017


Yes.  The current TS 17961 definition for vulnerability is:

Vulnerability
Set of conditions that allows an attacker to violate an explicit or
implicit security policy

rCs

On Fri, Jan 6, 2017 at 12:10 PM, Barton Miller <bart at cs.wisc.edu> wrote:

> "Flaw" is certainly a reasonable synonym for weakness.  (weakness == flaw
> == bug, for most folks)
>
> I think that the key is to *not *use vulnerability where we are talking
> about flaws in the code, unless it is associated with a specific exploit.
>
> --bart
>
> On 1/6/2017 11:04 AM, Robert Seacord wrote:
>
> On the subject(s) of vocabulary and free standards, it is preferable to
> use standard ISO definitions in ISO documents.
>
> The ISO/IEC 2382:2015(en) Information technology — Vocabulary is actually
> available for free from here:
>
> https://www.iso.org/obp/ui/#iso:std:iso-iec:2382:ed-1:v1:en
>
> If we can't find a definition we are looking for, we can add one.  TS
> 17961 added the following definitions:
>
> https://gitlab.com/trustable/C_Safety_and_Security_Rules_
> Study_Group/wikis/terms-and-definitions
>
> Instead of "weaknesses", I think we used the following term:
>
> Security flaw
> Defect that poses a potential security risk
>
> We can certainly revisit this definition, as it is lacking form a safety
> perspective. We can also include definitions form normative references,
> which we do here:  https://gitlab.com/trustable/
> C_Safety_and_Security_Rules_Study_Group/wikis/normative-references
>
> Another good source of definitions might be WG23:
> http://www.open-std.org/JTC1/SC22/WG23/docs/documents
>
> There are also some very lovely documents up there such as
>
> TR 24772-3 C Specific Language Vulnerabilities with Concurrency, submitted
> by David Keaton
> <http://www.open-std.org/JTC1/SC22/WG23/docs/ISO-IECJTC1-SC22-WG23_N0682-tr24772-3-post-meeting-46-with-concurrency-edits.docx>
>
> Which may be a great source for C specific language vulnerabilities with
> Concurrency (just a guess).
>
> If we would prefer to use terms defined outside of ISO/IEC documents, I
> think it is probably best that we just add them to our document.  In
> fact, I just went ahead and added this to the wiki:
>
> Safety flaw
> Defect that poses a potential safety risk
>
> It was fun and easy!  Everyone should try this.  8^)  The benefit of
> changing things in the wiki (or wherever, pending instructions from
> Laurence and Chris Polin) is that if no one ever changes this it will end
> up in the International Standard. Stuff posted to the list doesn't end up
> anywhere except on the list.
>
> Thanks,
> rCs
>
> On Fri, Jan 6, 2017 at 11:26 AM, Barton Miller via
> C-safe-secure-studygroup <c-safe-secure-studygroup at lists.trustable.io>
> wrote:
>
>> +1
>>
>> Good to use the accepted terminology.  (Though I have to admit, when
>> speaking, I often use the wrong term!)
>>
>> --bart
>>
>> On 1/6/2017 10:15 AM, Joe Jarzombek via C-safe-secure-studygroup wrote:
>>
>> All –
>>
>>
>>
>> We are actually addressing ‘weaknesses’; not just ‘vulnerabilities’ in
>> the C language.  Unless there is a specific exploit identified that takes
>> advantage of the flaw, it is only a weakness (which could be a quality or
>> security weakness).  If an exploit is identified that takes advantage of
>> the flaw, then that flaw is a vulnerability.
>>
>>
>>
>> Alternative statement:  *The existence (even if only theoretical) of an
>> exploit designed to take advantage of a weakness
>> <https://capec.mitre.org/about/glossary.html> (or multiple weaknesses) and
>> achieve a negative technical impact
>> <https://capec.mitre.org/about/glossary.html> is what makes a weakness a
>> vulnerability <https://capec.mitre.org/about/glossary.html>.*
>>
>>
>>
>> Derived from definitions in ITU-T CYBEX 1500 series and the MITRE
>> websites for CWE, CVE and CAPEC:
>>
>>
>>
>> *Weakness:  *mistake or flaw condition in architecture, design, code, or
>> process that, if left unaddressed, could under the proper conditions
>> contribute to a *cyber-enabled capability
>> <https://capec.mitre.org/about/glossary.html>* being vulnerable to
>> exploitation; represents potential source vectors for zero-day exploits --
>> Common Weakness Enumeration (CWE) *https://cwe.mitre.org/
>> <https://cwe.mitre.org/>*
>>
>> *Vulnerability: *mistake in software that can be directly used by a
>> hacker to gain access to a system or network; *Exposure: *configuration
>> issue of a mistake in logic that allows unauthorized access or exploitation
>> – Common Vulnerability and Exposure (CVE) *https://cve.mitre.org/
>> <https://cve.mitre.org/>*
>>
>> *Exploit: *take advantage of a weakness (or multiple weaknesses) to
>> achieve a *negative technical impact
>> <https://capec.mitre.org/about/glossary.html>* -- attack approaches from
>> the set of known exploits are used in the Common Attack Pattern Enumeration
>> and Classification (CAPEC) *https://capec.mitre.org
>> <https://capec.mitre.org/>*
>>
>>
>>
>> CVE, CWE, & CAPEC are part of the ITU-T CYBEX 1500 series & USG SCAP; so
>> the distinction and relationship between weaknesses and vulnerabilities is
>> already in standards.  Let’s at least be consistent in our terminology with
>> relevant standards.
>>
>>
>>
>> Regards,
>>
>>
>>
>>    -Joe -
>>
>>
>>
>> *Joe Jarzombek, **CSSLP*
>>
>> Global Manager, Software Supply Chain Solutions
>>
>> Email: joe.jarzombek at synopsys.com  |  Mobile: 703 627-4644
>> <%28703%29%20627-4644>
>>
>> Synopsys Software Integrity Group  |  www.synopsys.com/software  [image:
>> cid:image002.jpg at 01D23376.09F8BF20]
>> <https://www.linkedin.com/company/synopsys-software-integrity>  [image:
>> cid:image003.jpg at 01D23376.09F8BF20]
>> <https://www.facebook.com/SynopsysSoftwareIntegrity/>  [image:
>> cid:image004.jpg at 01D23376.09F8BF20] <https://twitter.com/sw_integrity>
>>
>> [image: SynopsysSiliconToSoftware]
>>
>> See my editorial in CrossTalk Sponsor’s Note on Supply Chain Risk in
>> Critical Infrastructure http://bit.ly/2cb6OHU
>>
>> See my presentation at AppSec USA --
>>
>> *Software Supply Chain Lifecycle Management:  Reducing Attack Vectors and
>> Enabling Rugged DevOps*
>>
>> *http://go.coverity.com/rs/157-LQW-289/images/SW%20SC20Lifecycle%20Management_AppSecUSA_13Oct2016.pdf
>> <http://go.coverity.com/rs/157-LQW-289/images/SW%20SC20Lifecycle%20Management_AppSecUSA_13Oct2016.pdf>*
>>
>>
>>
>> *From:* C-safe-secure-studygroup [mailto:c-safe-secure-studygro
>> up-bounces at lists.trustable.io
>> <c-safe-secure-studygroup-bounces at lists.trustable.io>] *On Behalf Of *Ian
>> Hawkes via C-safe-secure-studygroup
>> *Sent:* Friday, January 06, 2017 4:39 AM
>> *To:* Robert Seacord <rcseacord at gmail.com> <rcseacord at gmail.com>; C
>> Safety and Security Study Group Discussion <c-safe-secure-studygroup at list
>> s.trustable.io> <c-safe-secure-studygroup at lists.trustable.io>
>> *Subject:* Re: [C-safe-secure-studygroup] Static vs dynamic analysis:
>> who cares?
>>
>>
>>
>> I may well be wrong in my opinion here, so please forgive me if you think
>> I am but it seems opinions are being expressed at the moment. The root
>> topic of this group should be the C language as a language, not the testing
>> (static, dynamic or otherwise). That should come as a by-product of the
>> core activities.
>>
>>
>>
>> I would like to suggest that the group focus on the core language and the
>> vulnerabilities of the language that can manifest as safety or security
>> issues.
>>
>> The detailing of the issues can explain the what’s, whys and wherefores
>> of the language vulnerability. The ability to detect the vulnerability can
>> then be expressed as currently expected to be detected via static analysis
>> or through dynamic analysis is a sub section of the vulnerability
>> explanation. That way the document can be used as a guideline for how to
>> use the C language in an application (architecture agnostic) and as a
>> guidance as to where within the development process the vulnerabilities can
>> be checked for. If new methods of detection come about that are universally
>> accepted as sound, the guidelines section for that vulnerability can be
>> up-revisioned to accommodate. This should not need to occur very often.
>>
>> The safety processes such as ISO26262 (automotive) are not prescriptive
>> about language but focus more on the process of using the languages.
>> Security processes such as SDL are also very process focused.
>>
>> Each industry and company that could be using this document will need to
>> apply it to their individual development process that is steered / directed
>> or flavoured by their specific industry guidelines / rules or mandates.
>>
>> Keeping the document focused on the language and how the core elements
>> can produce “issues” and how these issues may manifest themselves would
>> allow tools vendors to concentrate on producing tools that are useful for
>> businesses and allow the businesses to concentrate on how to apply the
>> language to their products / processes / industries etc.
>>
>> Polluting the document with too many objectives will not solve anything
>> but will merely make the document less favoured by industry. Keep it simple
>> and clear with the language at its heart and specific attributes around the
>> elements of the language and you will find more businesses will make use of
>> the document.
>>
>> In my humble opinion that is what we should be trying to achieve here, a
>> document that  is accepted by industry as a useful and applicable core
>> guide so that it is accepted and used. Otherwise it is a document just for
>> the sake of a document.
>>
>>
>>
>> Mit freundlichen Grüßen \ Kind regards,
>>
>>
>>
>> Ian Hawkes
>>
>> *From:* C-safe-secure-studygroup [mailto:c-safe-secure-studygro
>> up-bounces at lists.trustable.io
>> <c-safe-secure-studygroup-bounces at lists.trustable.io>] *On Behalf Of *Robert
>> Seacord via C-safe-secure-studygroup
>> *Sent:* 05 January 2017 19:27
>> *To:* Roberto Bagnara <bagnara at cs.unipr.it>; C Safety and Security Study
>> Group Discussion <c-safe-secure-studygroup at lists.trustable.io>
>> *Subject:* Re: [C-safe-secure-studygroup] Static vs dynamic analysis:
>> who cares?
>>
>>
>>
>> This appears to be a reasonable and a sound argument.  I think my biggest
>> concern is as follow (so I would like to hear your views on this):
>>
>>
>>
>> 1. This group should involve analyzer vendors, as analyzer vendors are
>> the ones who will conform to the standard, and our consequently our primary
>> audience.
>>
>> 2. We are hoping that analyzer vendors will want to conform to the
>> standard, because they will see this as a market opportunity.
>>
>> 3. Analyzer vendors could claim partial conformance but would prefer to
>> claim conformance, as partial conformance is fairly meaningless (I'm sure I
>> could claim partial conformance with grep).
>>
>> 4. Different analysis techniques are more or less capable of diagnosing
>> different classes of problems.
>>
>> 5. Consequently, it will be difficult for a number of vendors,
>> implementing different analysis techniques, to agree on a non-trivial rule
>> set which they all feel they can enforce.
>>
>>
>>
>> What do you think?
>>
>>
>>
>> rCs
>>
>>
>>
>> On Thu, Jan 5, 2017 at 1:51 PM, Roberto Bagnara via
>> C-safe-secure-studygroup <c-safe-secure-studygroup at lists.trustable.io>
>> wrote:
>>
>> Hi there.
>>
>> I think that the current discussion on static analysis vs dynamic analysis
>> is a distraction from (my perception) of the interests of the study group.
>> In this writing, I will try to explain why.
>>
>> Traditionally, a "static analysis" of a program is any analysis that
>> can be conducted on the program without executing it.  And a "dynamic
>> analysis" of a program is any analysis that is based on the observation
>> of the behavior of a set of program executions.  This distinction
>> should be taken with care.  The crucial point is what it means to
>> execute a program:
>>
>> 1) the entire program can be executed on the intended target and the
>>    intended operational environment;
>> 2) parts of the program can be executed (with tests input generated in
>> some
>>    way) on the intended target and the intended operational environment;
>> 3) a more or less related program (e.g., compiled for a target different
>>    from the intended target) can be executed, entirely or in parts,
>>    on an instrumented environment that is different from the intended
>>    operational environment;
>> 4) in abstract interpretation, (parts of) a program can be abstractly
>>    executed on a domain of data descriptions;
>> 5) (parts of) a program can be symbolically executed.
>>
>> (Symbolic and concrete execution are mixed in so-called "concolic"
>> execution, but I do not want to overcomplicate things.)
>> I think there is no doubt that an analysis based on (1) is a dynamic
>> analysis, that (4) and (5) are techniques used in various static
>> analyzers, and that for (2) and (3) we would need more details in order
>> to understand whether we are talking about a static or a dynamic analysis.
>> The point, though, is: is this distinction really buying us anything?
>>
>> I believe it is unfortunate that TS 17961:2016 mentions "static
>> analysis" 12 times, sometimes in a way that seems to exclude tools
>> that use techniques that are based on some sort of (abstract, symbolic,
>> instrumented) execution of (parts of) the program.  My point is that,
>> for the scope of this working group, we should not care about slippery
>> distinctions about analysis techniques.  What we should care about
>> is automation.  To quote TS 17961:2016:
>>
>>     The purpose of this Technical Specification is to specify
>>     analyzable secure coding rules that can be automatically enforced
>>     to detect security flaws in C-conforming applications.
>>
>> Automatically enforceable, this is the key.  And another point I like
>> in TS 17961:2016 is in the "Scope" section:
>>
>>     This Technical Specification does not specify
>>       - the mechanism by which these rules are enforced [...]
>>       [...]
>>
>> I propose to focus on these points and to forget about the distinction
>> static analysys vs dynamic analysis.  What we want are automatic tools
>> where we push a button to start the analysis and, after a finite amount
>> of time (perhaps enforced by the provision of a stop button which
>> we can push when we want), they provide a set of results.
>> Such results might contain false positives and false negatives:
>> a conforming tool vendor will document, for each rule, whether
>> false positives and/or false negatives are possible, so that we know
>> whether compliance has been proved or if we have to do more work to
>> prove it.  The number of false positives and false negatives
>> is considered a quality of implementation issue by TS 17961:2016,
>> the idea being that the market will do what it takes to promote the
>> development of better tools.  Moreover, for a tool to be conforming
>> it is enough that "[f]or each rule, the analyzer reports a diagnostic
>> for at least one program that contains a violation of that rule."
>>
>> Let us take an example:
>>
>>    5.2 Accessing freed memory [accfree]
>>    Rule
>>    After an allocated block of dynamic storage has been deallocated by
>>    a memory management function, the evaluation of any pointers into
>>    the freed memory, including being dereferenced or acting as an
>>    operand of an arithmetic operation, type cast, or right-hand side
>>    of an assignment, shall be diagnosed.
>>
>> There are many techniques that allow to deal (to some extent) with this
>> undecidable rule:
>>
>> - abstract interpretation;
>> - (symbolic) model checking;
>> - random generation of test inputs;
>> - constraint-based generation of test inputs;
>> - language subsetting (e.g., MISRA C) and static analysis to prove
>>   the program is in the subset;
>> - valgrind;
>> - ...
>>
>> Consider the use of valgrind: I run (a variant of) the program
>> under valgrind for 10 minutes, by automatically generating inputs
>> any way I like.  This technique is able to detect violations of
>> [accfree] in a finite amount of time.  It has false negatives
>> (of course) but (depending on the variant and on the generated inputs)
>> very few or no false positives.  All in all it may be better than
>> many tools out there: why excluding it a priori?
>>
>> My point is that our worry should be that the rule is specified in
>> unambiguous terms, so that it is amenable to automatic cheking.  By
>> this I mean, that the specification should be precise enough that
>> consensus can be reached on whether a diagnostic given (resp., not
>> given) by a tool has to be considered or not a false positive (resp.,
>> a false negative).  This is not easy, but it is crucial.
>> Distinguishing the automatic analysis technique(s) employed (static,
>> dynamic, you-name-it) is, instead, irrelevant.
>> Kind regards,
>>
>>    Roberto
>>
>> --
>>      Prof. Roberto Bagnara
>>
>> Applied Formal Methods Laboratory - University of Parma, Italy
>> mailto:bagnara at cs.unipr.it
>>                               BUGSENG srl - http://bugseng.com
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__bugseng.com&d=DgMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=uxVVCWR8UcUyeakFZXcD5bxiMa6Nx1hPouclYdJYz00&m=yidvjZb100BG4skYC17Wt_J5Iq86p4POFm34SpoYeJg&s=RnueYp3RHHKcLzinoBDl5AO3XZpXwfgALKh0m7IkO8c&e=>
>>                               mailto:roberto.bagnara at bugseng.com
>>
>> _______________________________________________
>> C-safe-secure-studygroup mailing list
>> C-safe-secure-studygroup at lists.trustable.io
>> https://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-
>> secure-studygroup
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.trustable.io_cgi-2Dbin_mailman_listinfo_c-2Dsafe-2Dsecure-2Dstudygroup&d=DgMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=uxVVCWR8UcUyeakFZXcD5bxiMa6Nx1hPouclYdJYz00&m=yidvjZb100BG4skYC17Wt_J5Iq86p4POFm34SpoYeJg&s=w_k3nPkTzvBU2lM4oyU22B5hfpFIujBrGPtMQQrITTQ&e=>
>>
>>
>>
>>
>> _______________________________________________
>> C-safe-secure-studygroup mailing listC-safe-secure-studygroup at lists.trustable.iohttps://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-secure-studygroup
>>
>> _______________________________________________ C-safe-secure-studygroup
>> mailing list C-safe-secure-studygroup at lists.trustable.io
>> https://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-
>> secure-studygroup
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/01d08ae1/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 821 bytes
Desc: not available
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/01d08ae1/attachment-0003.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 816 bytes
Desc: not available
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/01d08ae1/attachment-0004.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 51878 bytes
Desc: not available
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/01d08ae1/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 816 bytes
Desc: not available
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/01d08ae1/attachment-0005.jpe>


More information about the C-safe-secure-studygroup mailing list