[Cryptography] Verifying X.509 Verification - how about an updated PKITS?

[Nikos Mavrogiannopoulos]

On Sun, 2014-04-06 at 12:25 -0400, Peter Trei wrote:
> We've now seen critical errors in two different TLS implementations, 
> both of which centered around (different) failures to properly 
> verify X.509 certificate chains. 

> As problematic as PKI is, these are bugs that shouldn't have happened.
> But they're both 'Type II' errors; a failure to recognize a problem
> that
> 
> should have generated an alarm. The 'happy path' code gets tested
> all the time - normal operation relies on it working. It's the
> 'unhappy
> path' which only gets exercised when something is going wrong that
> implements these holes. Testing the 'unhappy path' is crucial, 
> since normal operation doesn't run that way. Apparently this wasn't
> done.

Indeed, and that's an issue. Unfortunately the unhappy path isn't a
closed set of possibilities, so I doubt that any deterministic test
could significantly help. 

> I found myself thinking about a test suite of certificates which had
> known problems, and which *should* cause apps encountering them
> to throw errors. Never one to re-invent the wheel, I started poking
>  around.
> What I found was PKITS:
> http://csrc.nist.gov/groups/ST/crypto_apps_infra/pki/pkitesting.html
> Its about 10 years old.

And this suite (an older version than the current) was used to test the
verification routines of gnutls since long time; it did help with
expected failures, but did few for unexpected failures. The approach
shown in Victor's link or another fuzzying approach is more promising in
that respect.

regards,
Nikos