Is finding security holes a good idea?

Eric Rescorla ekr at rtfm.com
Wed Jun 16 17:12:18 EDT 2004


Thor Lancelot Simon <tls at rek.tjls.com> writes:
> On Tue, Jun 15, 2004 at 09:37:42PM -0700, Eric Rescorla wrote:
> If you won't grant that humans experienced in a given field tend to think
> in similar ways, fine.  We'll just have to agree to disagree; but I think
> you'll have a hard time making your case to anyone who _does_ believe that,
> which I think is most people.  If you do grant it, I think it behooves you
> to explain why you don't believe that's the case as regards finding bugs;
> or to withdraw your original claim, which is contingent upon it.

I'm sorry, but I don't think this follows at all.

Let's assume for the sake of argument that two people auditing
the same code section will find the same set of bugs. So, how
to account for the fact that obvious errors persist for long
periods of time in popular code bases? It must be that those
sections were never properly audited, since by hypothesis
the bugs are obvious and yet were not found. However, this
happens fairly often, which suggests that coverage must
be pretty bad. Accordingly, it's easy to see how you could
get low re-finding rates even if people roughly think alike.

Now, you could argue that because people think alike, everyone
looks at the exact same sections of the code, but I think
that this is belied by the fact that many of these self-same
obvious bugs are found in obvious places, such as protocol
parsers. 

So, while I think it's almost certainly not true that bug finding
order is completely random, I think it's quite plausible that it's
mostly random. Ultimately, however, it's an empirical question and I'd
be quite interested in seeing some studies on it.

I think I've said enough on this general topic. If you'd like to have
the last word, feel free.

-Ekr


---------------------------------------------------------------------
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to majordomo at metzdowd.com



More information about the cryptography mailing list