[Cryptography] Verifying X.509 Verification - how about an updated PKITS?

Peter Gutmann pgut001 at cs.auckland.ac.nz
Sun Apr 6 21:52:54 EDT 2014


Peter Trei <petertrei at gmail.com> writes:

>The 'happy path' code gets tested all the time - normal operation relies on
>it working. 

Yup.  The goal for any certificate use is "make it work, dammit!", since
they're already hard enough to deal with as it is.  After fighting this for
many years in my code I eventually introduced the concept of compliance
levels, ranging from "full PKIX" through to "oblivious".  There are a number
of public CAs that require the compliance level to be taken all the way down
to "oblivious" in order for their cert chains to work.  Conversely, some years
ago when I tested this, setting the compliance level to "full PKIX" rejected
ten out of eleven certs chains from public CAs.

>What I found was PKITS:
>http://csrc.nist.gov/groups/ST/crypto_apps_infra/pki/pkitesting.html Its
>about 10 years old.
>
>Reading the tests and related documents, this seems a pretty reasonable set
>of tests. It even includes a set of certs which *should* generate failures.

It's a somewhat dubious set of tests that range from reasonable through to
just plain stoopid.  I've commented on these before, he's one summary:

  (The US National Institute of Standards (NIST) has created a suite of test
  certificates that exercise various features of standards, but these function
  more as an exhaustive enumeration of standards-document features than a
  rigorous compliance-test suite.  As a result many of the problems discussed
  above are never checked, but on the other hand obscure aspects of standards
  present only for historical or political reasons are.  Since a number of
  these design artefacts are never used in real life, implementations have to
  add special support for them just to pass the tests, to the extent that some
  have added a special NIST-test mode that allows them to clear the tests,
  after which normal operation can be resumed at the flip of a compile option.
  The most extreme case of this reportedly detects the presence of the NIST
  test certificates and returns the expected results from an internal lookup
  table, which saves the developers the trouble of having to implement the
  complex certificate-processing functionality that would be required to pass
  the tests. These sorts of developer tricks are another example of gaming
  onerous qualification tests that’s discussed in “Asking the Drunk Whether
  He’s Drunk” on page 54.  What makes this even more entertaining is the fact
  that some versions of the test suite contained errors and yet
  implementations still successfully processed it, indicating that the
  implementation target was “whatever’s required to pass the test” rather than
  “what the standard requires”).

Peter.


More information about the cryptography mailing list