[C-safe-secure-studygroup] Static vs dynamic analysis: who cares?
Joe Jarzombek
Joe.Jarzombek at synopsys.com
Fri Jan 6 16:15:57 UTC 2017
All –
We are actually addressing ‘weaknesses’; not just ‘vulnerabilities’ in the C language. Unless there is a specific exploit identified that takes advantage of the flaw, it is only a weakness (which could be a quality or security weakness). If an exploit is identified that takes advantage of the flaw, then that flaw is a vulnerability.
Alternative statement: The existence (even if only theoretical) of an exploit designed to take advantage of a weakness<https://capec.mitre.org/about/glossary.html> (or multiple weaknesses) and achieve a negative technical impact<https://capec.mitre.org/about/glossary.html> is what makes a weakness a vulnerability<https://capec.mitre.org/about/glossary.html>.
Derived from definitions in ITU-T CYBEX 1500 series and the MITRE websites for CWE, CVE and CAPEC:
Weakness: mistake or flaw condition in architecture, design, code, or process that, if left unaddressed, could under the proper conditions contribute to a cyber-enabled capability<https://capec.mitre.org/about/glossary.html> being vulnerable to exploitation; represents potential source vectors for zero-day exploits -- Common Weakness Enumeration (CWE) https://cwe.mitre.org/
Vulnerability: mistake in software that can be directly used by a hacker to gain access to a system or network; Exposure: configuration issue of a mistake in logic that allows unauthorized access or exploitation – Common Vulnerability and Exposure (CVE) https://cve.mitre.org/
Exploit: take advantage of a weakness (or multiple weaknesses) to achieve a negative technical impact<https://capec.mitre.org/about/glossary.html> -- attack approaches from the set of known exploits are used in the Common Attack Pattern Enumeration and Classification (CAPEC) https://capec.mitre.org<https://capec.mitre.org/>
CVE, CWE, & CAPEC are part of the ITU-T CYBEX 1500 series & USG SCAP; so the distinction and relationship between weaknesses and vulnerabilities is already in standards. Let’s at least be consistent in our terminology with relevant standards.
Regards,
-Joe -
Joe Jarzombek, CSSLP
Global Manager, Software Supply Chain Solutions
Email: joe.jarzombek at synopsys.com<mailto:joe.jarzombek at synopsys.com> | Mobile: 703 627-4644
Synopsys Software Integrity Group | www.synopsys.com/software<http://www.synopsys.com/software> [cid:image002.jpg at 01D23376.09F8BF20] <https://www.linkedin.com/company/synopsys-software-integrity> [cid:image003.jpg at 01D23376.09F8BF20] <https://www.facebook.com/SynopsysSoftwareIntegrity/> [cid:image004.jpg at 01D23376.09F8BF20] <https://twitter.com/sw_integrity>
[SynopsysSiliconToSoftware]
See my editorial in CrossTalk Sponsor’s Note on Supply Chain Risk in Critical Infrastructure http://bit.ly/2cb6OHU
See my presentation at AppSec USA --
Software Supply Chain Lifecycle Management: Reducing Attack Vectors and Enabling Rugged DevOps
http://go.coverity.com/rs/157-LQW-289/images/SW%20SC20Lifecycle%20Management_AppSecUSA_13Oct2016.pdf
From: C-safe-secure-studygroup [mailto:c-safe-secure-studygroup-bounces at lists.trustable.io] On Behalf Of Ian Hawkes via C-safe-secure-studygroup
Sent: Friday, January 06, 2017 4:39 AM
To: Robert Seacord <rcseacord at gmail.com>; C Safety and Security Study Group Discussion <c-safe-secure-studygroup at lists.trustable.io>
Subject: Re: [C-safe-secure-studygroup] Static vs dynamic analysis: who cares?
I may well be wrong in my opinion here, so please forgive me if you think I am but it seems opinions are being expressed at the moment. The root topic of this group should be the C language as a language, not the testing (static, dynamic or otherwise). That should come as a by-product of the core activities.
I would like to suggest that the group focus on the core language and the vulnerabilities of the language that can manifest as safety or security issues.
The detailing of the issues can explain the what’s, whys and wherefores of the language vulnerability. The ability to detect the vulnerability can then be expressed as currently expected to be detected via static analysis or through dynamic analysis is a sub section of the vulnerability explanation. That way the document can be used as a guideline for how to use the C language in an application (architecture agnostic) and as a guidance as to where within the development process the vulnerabilities can be checked for. If new methods of detection come about that are universally accepted as sound, the guidelines section for that vulnerability can be up-revisioned to accommodate. This should not need to occur very often.
The safety processes such as ISO26262 (automotive) are not prescriptive about language but focus more on the process of using the languages. Security processes such as SDL are also very process focused.
Each industry and company that could be using this document will need to apply it to their individual development process that is steered / directed or flavoured by their specific industry guidelines / rules or mandates.
Keeping the document focused on the language and how the core elements can produce “issues” and how these issues may manifest themselves would allow tools vendors to concentrate on producing tools that are useful for businesses and allow the businesses to concentrate on how to apply the language to their products / processes / industries etc.
Polluting the document with too many objectives will not solve anything but will merely make the document less favoured by industry. Keep it simple and clear with the language at its heart and specific attributes around the elements of the language and you will find more businesses will make use of the document.
In my humble opinion that is what we should be trying to achieve here, a document that is accepted by industry as a useful and applicable core guide so that it is accepted and used. Otherwise it is a document just for the sake of a document.
Mit freundlichen Grüßen \ Kind regards,
Ian Hawkes
From: C-safe-secure-studygroup [mailto:c-safe-secure-studygroup-bounces at lists.trustable.io] On Behalf Of Robert Seacord via C-safe-secure-studygroup
Sent: 05 January 2017 19:27
To: Roberto Bagnara <bagnara at cs.unipr.it<mailto:bagnara at cs.unipr.it>>; C Safety and Security Study Group Discussion <c-safe-secure-studygroup at lists.trustable.io<mailto:c-safe-secure-studygroup at lists.trustable.io>>
Subject: Re: [C-safe-secure-studygroup] Static vs dynamic analysis: who cares?
This appears to be a reasonable and a sound argument. I think my biggest concern is as follow (so I would like to hear your views on this):
1. This group should involve analyzer vendors, as analyzer vendors are the ones who will conform to the standard, and our consequently our primary audience.
2. We are hoping that analyzer vendors will want to conform to the standard, because they will see this as a market opportunity.
3. Analyzer vendors could claim partial conformance but would prefer to claim conformance, as partial conformance is fairly meaningless (I'm sure I could claim partial conformance with grep).
4. Different analysis techniques are more or less capable of diagnosing different classes of problems.
5. Consequently, it will be difficult for a number of vendors, implementing different analysis techniques, to agree on a non-trivial rule set which they all feel they can enforce.
What do you think?
rCs
On Thu, Jan 5, 2017 at 1:51 PM, Roberto Bagnara via C-safe-secure-studygroup <c-safe-secure-studygroup at lists.trustable.io<mailto:c-safe-secure-studygroup at lists.trustable.io>> wrote:
Hi there.
I think that the current discussion on static analysis vs dynamic analysis
is a distraction from (my perception) of the interests of the study group.
In this writing, I will try to explain why.
Traditionally, a "static analysis" of a program is any analysis that
can be conducted on the program without executing it. And a "dynamic
analysis" of a program is any analysis that is based on the observation
of the behavior of a set of program executions. This distinction
should be taken with care. The crucial point is what it means to
execute a program:
1) the entire program can be executed on the intended target and the
intended operational environment;
2) parts of the program can be executed (with tests input generated in some
way) on the intended target and the intended operational environment;
3) a more or less related program (e.g., compiled for a target different
from the intended target) can be executed, entirely or in parts,
on an instrumented environment that is different from the intended
operational environment;
4) in abstract interpretation, (parts of) a program can be abstractly
executed on a domain of data descriptions;
5) (parts of) a program can be symbolically executed.
(Symbolic and concrete execution are mixed in so-called "concolic"
execution, but I do not want to overcomplicate things.)
I think there is no doubt that an analysis based on (1) is a dynamic
analysis, that (4) and (5) are techniques used in various static
analyzers, and that for (2) and (3) we would need more details in order
to understand whether we are talking about a static or a dynamic analysis.
The point, though, is: is this distinction really buying us anything?
I believe it is unfortunate that TS 17961:2016 mentions "static
analysis" 12 times, sometimes in a way that seems to exclude tools
that use techniques that are based on some sort of (abstract, symbolic,
instrumented) execution of (parts of) the program. My point is that,
for the scope of this working group, we should not care about slippery
distinctions about analysis techniques. What we should care about
is automation. To quote TS 17961:2016:
The purpose of this Technical Specification is to specify
analyzable secure coding rules that can be automatically enforced
to detect security flaws in C-conforming applications.
Automatically enforceable, this is the key. And another point I like
in TS 17961:2016 is in the "Scope" section:
This Technical Specification does not specify
- the mechanism by which these rules are enforced [...]
[...]
I propose to focus on these points and to forget about the distinction
static analysys vs dynamic analysis. What we want are automatic tools
where we push a button to start the analysis and, after a finite amount
of time (perhaps enforced by the provision of a stop button which
we can push when we want), they provide a set of results.
Such results might contain false positives and false negatives:
a conforming tool vendor will document, for each rule, whether
false positives and/or false negatives are possible, so that we know
whether compliance has been proved or if we have to do more work to
prove it. The number of false positives and false negatives
is considered a quality of implementation issue by TS 17961:2016,
the idea being that the market will do what it takes to promote the
development of better tools. Moreover, for a tool to be conforming
it is enough that "[f]or each rule, the analyzer reports a diagnostic
for at least one program that contains a violation of that rule."
Let us take an example:
5.2 Accessing freed memory [accfree]
Rule
After an allocated block of dynamic storage has been deallocated by
a memory management function, the evaluation of any pointers into
the freed memory, including being dereferenced or acting as an
operand of an arithmetic operation, type cast, or right-hand side
of an assignment, shall be diagnosed.
There are many techniques that allow to deal (to some extent) with this
undecidable rule:
- abstract interpretation;
- (symbolic) model checking;
- random generation of test inputs;
- constraint-based generation of test inputs;
- language subsetting (e.g., MISRA C) and static analysis to prove
the program is in the subset;
- valgrind;
- ...
Consider the use of valgrind: I run (a variant of) the program
under valgrind for 10 minutes, by automatically generating inputs
any way I like. This technique is able to detect violations of
[accfree] in a finite amount of time. It has false negatives
(of course) but (depending on the variant and on the generated inputs)
very few or no false positives. All in all it may be better than
many tools out there: why excluding it a priori?
My point is that our worry should be that the rule is specified in
unambiguous terms, so that it is amenable to automatic cheking. By
this I mean, that the specification should be precise enough that
consensus can be reached on whether a diagnostic given (resp., not
given) by a tool has to be considered or not a false positive (resp.,
a false negative). This is not easy, but it is crucial.
Distinguishing the automatic analysis technique(s) employed (static,
dynamic, you-name-it) is, instead, irrelevant.
Kind regards,
Roberto
--
Prof. Roberto Bagnara
Applied Formal Methods Laboratory - University of Parma, Italy
mailto:bagnara at cs.unipr.it<mailto:bagnara at cs.unipr.it>
BUGSENG srl - http://bugseng.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__bugseng.com&d=DgMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=uxVVCWR8UcUyeakFZXcD5bxiMa6Nx1hPouclYdJYz00&m=yidvjZb100BG4skYC17Wt_J5Iq86p4POFm34SpoYeJg&s=RnueYp3RHHKcLzinoBDl5AO3XZpXwfgALKh0m7IkO8c&e=>
mailto:roberto.bagnara at bugseng.com<mailto:roberto.bagnara at bugseng.com>
_______________________________________________
C-safe-secure-studygroup mailing list
C-safe-secure-studygroup at lists.trustable.io<mailto:C-safe-secure-studygroup at lists.trustable.io>
https://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-secure-studygroup<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.trustable.io_cgi-2Dbin_mailman_listinfo_c-2Dsafe-2Dsecure-2Dstudygroup&d=DgMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=uxVVCWR8UcUyeakFZXcD5bxiMa6Nx1hPouclYdJYz00&m=yidvjZb100BG4skYC17Wt_J5Iq86p4POFm34SpoYeJg&s=w_k3nPkTzvBU2lM4oyU22B5hfpFIujBrGPtMQQrITTQ&e=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/b9ebce2b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image009.jpg
Type: image/jpeg
Size: 816 bytes
Desc: image009.jpg
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/b9ebce2b/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image010.jpg
Type: image/jpeg
Size: 816 bytes
Desc: image010.jpg
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/b9ebce2b/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image011.jpg
Type: image/jpeg
Size: 821 bytes
Desc: image011.jpg
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/b9ebce2b/attachment-0005.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image012.png
Type: image/png
Size: 51878 bytes
Desc: image012.png
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170106/b9ebce2b/attachment-0001.png>
More information about the C-safe-secure-studygroup
mailing list