[C-safe-secure-studygroup] more on dynamic+static analysis

Tue Jan 31 20:08:04 UTC 2017

On Sun, Jan 29, 2017 at 8:16 PM, Wheeler, David A <dwheeler at ida.org> wrote:

> If we’re going to discuss lessons learned from Heartbleed, please take a
> peek at my extensive paper on exactly that topic, “How to Prevent the next
> Heartbleed”, https://www.dwheeler.com/essays/heartbleed.html ; I also
> published a brief summary of that in IEEE Computer. Volume 47, Issue 8.
> August 2014. pp. 80-83.
>
>
>
> It’s important to note that **both** static & dynamic analysis tools of
> the time missed Heartbleed, including static analysis tools AND fuzzing
> tools.
>

asan and valgrind were always capable of finding heartbleed if supplied
with a proper input; what was missing is the good input generator.
libFuzzer and AFL were conceived around the time of heartbleed, but have
not become widely used nor known yet.
But some heavier fuzzing systems were available long before heartbleed
(e.g. Microsoft's SAGE
<https://patricegodefroid.github.io/public_psfiles/ndss2008.pdf>, and
Google's internal fuzzing system <http://j00ru.vexillium.org/?p=2211>) and
were capable of finding it.
The problem was not the lack of the fuzzing tools, but lack of habit to use
them (and yes, nowadays the tools are better and easier to use).

The lack of habit is still the major problem -- developers are just not
used to the concept of fuzzing.
With many projects it takes us to report a dozen bugs to the code owners
before they get convinced to do fuzzing themselves.
It's like with unit testing 15 years ago. The concept was there, but most
developers did not use it. (many still don't)

>
>
> Fuzzing processes widely practiced at the time would not, and did not,
> discover Heartbleed.  A key problem is that Heartbleed was a buffer over *
> *read**, not a buffer over-*write*, and so most detection systems would
> not detect it.  There’s the additional problem that simple “just spew
> random data” fuzzers have trouble finding things in general.
>

Of course, an unguided fuzzer has very little chance here.

>
>
> Heartbleed **was**, of course, found by a fuzzer – namely Codenomicon’s.
>

> But in their case, they used generational (“smart”) fuzzing along with
> much more information about what behavior was expected – this was research
> at the time for their new techniques, and absolutely not standard at the
> time.  It’s still unusual, because you have to invest time to develop the
> detailed information about the protocol & expected results.
>
>
>
> Heartbleed can also be found using ASAN (which **can** detect over-reads)
> – and it’s especially easy if you pair it with a coverage-based fuzzer
> (like American Fuzzy Lop).
>
>
>
> In practice you need to combine both static & dynamgiic approaches.  It’s
> not clear to me that it’s best to combine their requirements in a **spec**,
> but that’s a different question.
>
>
>
> For more information, see the paper.
>

nice!

--kcc

>
>
> --- David A. Wheeler
>
>
>
>
>
>
>
> *From:* C-safe-secure-studygroup [mailto:c-safe-secure-studygro
> up-bounces at lists.trustable.io] *On Behalf Of *Kostya Serebryany
> *Sent:* Friday, January 27, 2017 5:56 PM
> *To:* C Safety and Security Study Group Discussion
> *Subject:* Re: [C-safe-secure-studygroup] more on dynamic+static analysis
>
>
>
>
>
>
>
> On Fri, Jan 27, 2017 at 2:53 PM, Barton Miller <bart at cs.wisc.edu> wrote:
>
> I'm the last person to say anything negative about fuzzing.  But even to
> fuzz to detect Heartbleed, you have to know that you're creating a valid
> heartbeat packet with random payload length.
>
> Hmm. No. Why?
>
> Take a look at https://github.com/google/fuzzer-test-suite/blob/master/o
> penssl-1.0.1f/target.cc
>
> (which is how heartbleed can be found).
>
> It does not require anything about 'valid' packets.
>
>
>
>
>
>
>
> --bart
>
>
>
> On 1/27/2017 4:33 PM, Kostya Serebryany wrote:
>
>
>
>
>
> On Fri, Jan 27, 2017 at 7:18 AM, Barton Miller <bart at cs.wisc.edu> wrote:
>
> I would like to add one factor to this discussion of static vs. dynamic:
> the necessary test cases for dynamic testing.
>
> It is true that you can add dynamic checks to the cases below and would
> catch violations.  That was also true of the Heartbleed OpenSSL
> vulnerability.  However no one had thought of testing OpenSSL under the
> attack scenario.
>
> So, yes, dynamic testing can catch violations, especially of these
> memory-out-bounds type problems, but only if paired with the right input
> test cases.
>
> That's precisely my point! :)
>
> dynamic checks are fully useful only when coupled with fuzzing.
>
>
>
> Since the heartbleed, openssl and a few related projects started fuzzing
> very actively (see e.g. https://github.com/openss
> l/openssl/tree/master/fuzz)
>
> and found a few more bugs this way (last one was disclosed yesterday:
> https://github.com/google/fuzzer-test-suite/tree/master/openssl-1.1.0c).
>
>
>
> heartbleed itself can be found by fuzzing in a couple of seconds:
>
> https://github.com/google/fuzzer-test-suite/tree/master/openssl-1.0.1f
>
>
>
>
>
> --kcc
>
> --bart
>
>
>
> On 1/25/2017 5:25 PM, Kostya Serebryany wrote:
>
> . . .
>
>
>
> // This can't be analyzed statically, but we can insert  checks that
>
> // will perform precise checks dynamically
>
> int a[10];
>
> extern int get_idx();  // No source available during analysis
>
> void foo() { a[get_idx()] = 0; }
>
>
>
> // again, not statically checkable, but we can insert a precise dynamic
> check
>
> void foo(int n, int m) {
>
>    int *p = (int*)malloc(n * sizeof(int));
>
>   ...
>
>   p[m] = 0;
>
>  }
>
>
> _______________________________________________
> C-safe-secure-studygroup mailing list
> C-safe-secure-studygroup at lists.trustable.io
> https://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-
> secure-studygroup
>
>
>
>
>
> _______________________________________________
>
> C-safe-secure-studygroup mailing list
>
> C-safe-secure-studygroup at lists.trustable.io
>
> https://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-secure-studygroup
>
>
>
>
> _______________________________________________
> C-safe-secure-studygroup mailing list
> C-safe-secure-studygroup at lists.trustable.io
> https://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-
> secure-studygroup
>
>
>
> _______________________________________________
> C-safe-secure-studygroup mailing list
> C-safe-secure-studygroup at lists.trustable.io
> https://lists.trustable.io/cgi-bin/mailman/listinfo/c-safe-
> secure-studygroup
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.trustable.io/pipermail/c-safe-secure-studygroup/attachments/20170131/07251919/attachment.html>