'Proving' the correctness of a network encryption system test system

John Denker jsd at av8n.com
Sun Dec 5 18:03:48 EST 2004


Fredrik Henbjork wrote:
> Alice has:
> 
> 1. A system which does processing of encrypted network streams.
> 
> Alice wants the following from Bob:
> 
> 2. A test system for the processing system in 1. This system is going
>  to be used to decide if the processing system in 1 is working 
> (processing) as it should.

This question is unanswerable.  The attempted answer by Iang
misses the point.

The point is that we haven't been told enough, not nearly
enough, about what the phrase "as it should" means.

Step Zero is for Bob to sit down with Alice and make a detailed
list of all the things Alice wants to happen, and an equally
detailed list of all the things Alice doesn't want to happen.

In my experience, it may be possible to meet Alice's objectives
with no encryption at all ... or it may be impossible to meet
Alices objectives using any known combination of techniques
(crypto included) ... or somewhere in between.

Most particularly, there is a huuuge difference between
  -- verifying that the "encrypted network stream" is correct, and
  -- ascertaining that the "system" is working "as it should".

To say that again in more standard terms, there is a huuge
difference between cryptography and security.

As a specific and not-very-far-fetched example, suppose the
"encrypted network streams" consist of SSH traffic flowing
to/from port 22 on each of Alice's machines, and are fully
ideal in the sense of being as secure as you could possibly
ask SSH to be.  Alas that does prevent *anybody* from
telnetting to port 23 and logging in as root.  Maybe Alice
wants to permit that, or maybe (probably) not.

The IPsec RFC, to its credit, addresses both edges of the
two-edged sword.  It specifies what the encryption should
look like, and it also requires the existence of an SPDB
(Security Policy DataBase) that has enough expressive
power to prevent a wide class of undesired traffic from
occuring.  Of course this leaves to the implementor the
question of what policy "should" be in the SPDB, but at
least the mechanism is there begging to be used.

> 3. A test system for the test system in 2. This system is going to
> be used to decide if the test system in 2 is working (testing) as
> it should.

This brings to mind Dykstra's dictum that testing can
prove the presence of bugs, but cannot prove the absence
of bugs.  You should not imagine even for a moment that
any amount of testing will demonstrate that the system
is working "as it should".

Therefore if the question is whether the test system is
proving the security of the main system, the meta-test
system called for in item 3 can be replaced by a simple
box that answers "no".

As a specific example, suppose a port-scan tells you that
port 23 is not open.  But things could be set up such that
the port is only open during certain hours of the day, and
you didn't happen to look at the right time.  (I've seen
stranger things.)

Bottom line:  Alice doesn't need just a test.  Alice needs
a whole lot of things, including good specs, good design,
design reviews, good implementation, implementation audits,
etc. etc. etc. ... and even then Alice will have to accept
a nonzero amount of residual risk.


---------------------------------------------------------------------
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to majordomo at metzdowd.com



More information about the cryptography mailing list