A slight modification of my comments on PKI.

[Stephan Neuhaus]

On Jul 29, 2010, at 22:23, Anne & Lynn Wheeler wrote:

> On 07/28/2010 10:34 PM, dan at geer.org wrote:
>> The design goal for any security system is that the number of
>> failures is small but non-zero, i.e., N>0.  If the number of
>> failures is zero, there is no way to disambiguate good luck
>> from spending too much.  Calibration requires differing outcomes.
>> Regulatory compliance, on the other hand, stipulates N==0 failures
>> and is thus neither calibratable nor cost effective.  Whether
>> the cure is worse than the disease is an exercise for the reader.
> 
> another design goal for any security system might be "security proportional to risk". 

Warning:  self-promotion (well, rather: project promotion) ahead.

This is exactly what we are trying to do in an EU project in which I'm involved. The project, called MASTER, is more concerned with regulatory compliance than security, even though security of course plays a large role.

The insight is that complex systems will probably never have N = 0 (in Dan's terms), so we will have to calibrate the controls so that the N becomes commensurate with the risk.  To do this, we have two main tools:

First, there is a methodology that describes in detail how to break down your high-level regulatory goals (which we call control objectives) into actionable pieces. This breakdown tells you exactly what you need to control, and how. It is controlled by risk analysis, so you can say at any point why you made certain decisions, and conversely, if a regulation changes, you know exactly which parts of your processes are affected (assuming the risk analysis doesn't have to be completely redone as part of the regulatory change).

Second, as part of this breakdown process, you define, for each broken-down control objective, indicators.  These are metrics that indicate (1) whether the process part you are currently looking at is  compliant (i.e., has low enough N), and (2) whether this low N is pure luck or the result of well-placed and correctly functioning controls.

One benefit of having indicators at every level of breakdown is that you get metrics that mean something *at this level*. For example, at the lowest level, you might get "number of workstations with outdated virus signatures", while at the top you might get "money spent in the last year on lawsuits asserting a breach of privacy". This forces one to do what Andrew Jaquith calls "contextualisation" in his book, and prevents the approach sadly taken by so many risk analysis papers, namely simply propagating "risk values" from the leaves of a risk tree to the root using some propagation rule, leaving the root with a beautifully computed, but sadly irrelevant, number. Another benefit is that if some indicator is out of some allowed band, the remedy will usually be obvious to a person working with that indicator. In other words, our indicators are actionable.

The question of whether the cure is worse than the disease can't be settled definitively by us.  We have done some evaluation of our approach, and preliminary results seem to indicate that users like it. (This is said with all the grains of salt usually associated with preliminary user studies.) How much it costs to deploy is unknown, since the result of our project will be a prototype rather than an industrial-strength product, but our approach allows you to deploy only parts.

Best,

Stephan
---------------------------------------------------------------------
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to majordomo at metzdowd.com