Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Badly needs static analysis. coverity is free for open source projects, will find many of the same issues, and produces reports that directly identify the buggy code, rather than the crash which might come thousands of instructions later in a completely different layer, requiring extensive reverse engineering to identify the source. And then when you've fixed one, the other test cases all need to be re-tested because they might have come from the same root cause, despite crashing in different locations.

Particularly when the hit rate is very high, fuzzing is a stupidly inefficient way to find bugs.



Needs both.

> rather than the crash which might [...]

Assert like mad. Break out your debug allocators, your asan, your checked iterators. Fuzzing can be as bad as you say, but it can also give you extremely easy to diagnose repro cases - and catch an incredibly wide range of issues for incredibly little work, in my experience.

Static analysis generally won't tell you "Hey, if I almost-but-not-quite exhaust your address space allocating to successfully parsing this file, I can get your next closed-source API call to segfault!" - useful for tracking down each and every possible allocation related to networking deserialization so you can configure sane quotas (such as "array size probably shouldn't be larger than bytes sent, bail out safely instead of trying to allocate that")

With most sane setups that come to mind, no false positives either.

> And then when you've fixed one, the other test cases all need to be re-tested because they might have come from the same root cause, despite crashing in different locations.

Your tooling should do this automatically for you. If it doesn't, get better tooling. AFL. SDL MiniFuzz. Whatever floats your boat.

> Particularly when the hit rate is very high, fuzzing is a stupidly inefficient way to find bugs.

I've had it be stupidly efficient in terms of programmer time once setup. Occasionally leaving my computer on overnight? Worth.


I've found coverity to be very hard to set up with a normal developer workflow on github. ==> I think I've given up on it 3 times.

Do you have any guidance around getting it to work?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: