[Cryptography] Testing crypto protocol implementations

[Viktor Dukhovni]

On Sun, Mar 02, 2014 at 03:06:24AM +1300, Peter Gutmann wrote:

> >Are there unit testing framworks for crypto libs? I suppose they would do as
> >Peter suggested below in depth running along side assertions and check
> >equalities.
> 
> I doubt there are, because to do this kind of testing you need to poke around
> deep inside the internals of the crypto library.  What you're doing is
> generating incorrect or malformed output in a controlled manner, which isn't
> generally something that's supported in standard code.  In fact you don't even
> want the capability to do this present in your code (in my case you need to do
> a custom build) because it's rather dangerous to have sitting in there.

This is a good point.  My view is that a tool that tests the security
of a protocol implementation, likely needs to be a separate highly
scriptable implementation of that protocol which can be programmed
to deviate from the protocol at every step, (modify and/or re-order
the expected protocol messages in controlled ways).

This would be a costly project for each implementation to develop.
Ideally such a tool would be available for testing multiple
implementations, but it is far from clear how it would be funded.

Having lived and breathed DANE TLSA for the last year, and looked
at a bunch of flawed implementations, I can see the need for such
a test suite for DANE, in the form of test services with interesting
TLSA records and even more "interesting" certificate chains.

There are a handful of DANE TLSA test sites, but their "interesting"
combinations of certificate chains and TLSA records are far from
sufficiently comprehensive.  Lack of a reasonably comprehensive
test-bed almost assures that flawed implementations will continue
to be produced, and users will continue to use them unaware of
their defects.

-- 
	Viktor.