Windows 2000 declared secure

[Jonathan S. Shapiro]

On Sat, 2002-11-02 at 13:31, Adam Shostack wrote:
> On Sat, Nov 02, 2002 at 11:54:36AM -0500, Jonathan S. Shapiro wrote:
> | The effectiveness of
> | the levels is modestly exaggerated, and the importance of going for
> | higher levels is grossly understated.
> | 
> | One unfortunate consequence is that NSA has seen no need to publish
> | guidelines on performing higher-level evaluations, because their has
> | been no demand.
> 
> Could you define 'importance' here?  Given a lack of demand, what are
> you using as criteria?  How can we translate that into something
> that's important to buyers? Or otherwise convince the buyers of
> systems to demand better?  (Leading to NSA publishing those higher
> level eval guidelines, etc.)

Apologies once again for the length of my reply. The issues are both
complicated and political. I also need to preface this by saying that I
am an active participant in the dialog around this issue. My usual role
is poking sharp sticks into people's eyes, so please read what I have to
say skeptically.

Context: There are international mutual-recognition treaties covering
EAL4 and below, so if you get an EAL4 evaluation in Germany, it's
accepted as binding in the US. Above EAL4 there is no mutual
recognition. From talking to various assurance evaluators (current and
former) and also to people within NSA, the alleged rationale behind this
is in two parts:

(1) There is a perception that commercial products won't
    seek higher evaluation, so there has been no real pressure
    to push the treaties beyond this.
(2) There is a perception that products seeking assurance
    above EAL4 are likely to be targeted primarily at military
    and/or sensitive applications. No country wants to be in the
    position of being treaty-obligated to accept assurance from
    another country about militarily sensitive materials.

Given that an EAL4 certification can fairly be characterized as "nowhere
near good enough for serious commercial use today", I think it is fair
to harshly criticize these rationales as rather thin rationalizations.

I am (ahem) cynical. I suspect that the absence of treaty agreement may
also have something to do with the fact that as of today the NSA has not
published procedural guidelines for anything above EAL4. In fact, they
have not finished writing them. Brian Snow (NSA), who has the mandate
for publication of such guidelines in the U.S., is in a fairly absurd
position. On the one hand he goes around telling everybody "it's
important to do higher assurance", but on the other hand the current
state of affairs (which is his responsibility) makes this impossible.

There is another subtext. One agenda of the evaluation community is to
get people in the habit of doing evaluations before they raise the
stakes. In order for this strategy to work, products have to pass the
evaluations. The de facto effect is a desire not to set too high a bar.
I personally disagree with this strategy, but even if I did I would
argue that EAL4 is not a barrier to any current commodity operating
system, and the US national interest is not served so long as the best
standards are inadequate for daily commercial use. World interests
aren't served either, but the NSA mandate stops with the US.

When pressed, Brian Snow says that until somebody wants to actually do a
higher level evaluation, NSA cannot justify the expense of doing the
higher level guidelines, but that they could proceed with a higher
evaluation today using the older guidelines that were applied under
TCSEC. This is probably true, but it is not clear whether such an
evaluation would have any commercial value, because the quality of the
resulting evaluation is not characterized by a published standard of
practice. I am publicly on record as saying that EROS will attempt EAL7
evaluation as soon as possible after NSA published appropriate
evaluation guidelines.

This is one of the issues that I use sharp sticks on. Brian usually
takes them in the eye, and I think is privately glad that somebody is
agitating over this issue. More specifically, I have stated publicly
that the single greatest impediment to a successful EAL7 evaluation for
EROS is the absence of relevant guidelines. Without those guidelines, we
can neither sensibly prepare for evaluation nor know what the result
would mean. A lot of people out there who do secure OS work know about
our stuff and our formal verification work and find these statements
quite threatening. They know that (a) EROS can probably do it, and (b)
most of them cannot. In fairness, I should add that the conventional
approach to this has been for the customer to pay for the evaluation,
and Hopkins certainly isn't going to do this. Until this changes, I can
pretty much guarantee that no open source OS will go through a
high-level evaluation. Brian and I have talked intermittently about how
to solve this, and I think we have a strategy worked out for when the
time comes.


So the current state of affairs is that for levels above EAL4 the
evaluation must be performed with participants from the government of
the host country, and has no standing outside of the host country. I
suspect that someone achieving a successful EAL5 evaluation in the US
could claim EAL4 elsewhere under the treaties, but a US-achieved EAL5
would not qualify a product to be submitted on, say, a German military
solicitation requiring an EAL5 certification.

Note that the current policies have a curious effect. Companies pursue
evaluation primarily for the marketing benefit of the certification. The
minute they step from EAL4 to EAL5, they suddenly need to spend millions
of dollars **per country** and the marketing payoff stops. A consequence
is that higher levels of assurance work are not cost effective on the
development end. The de facto effect is to ensure that there is a strong
economic disincentive to make a truly secure operating system widely
available to the public sector. Given the historical export policies of
the US, the fact that secure systems are still considered sensitive by
the armed services, and the fact that B2 or better systems may still be
export controlled under ITAR (their status is ambiguous at present), one
must question whether this outcome is an accident or an intentional
result of undisclosed policy decisions by various groups within the U.S.
government.

Whatever policy NSA thinks is a good idea, the impact on the U.S.
commercial sector is very clear. Until there is a unified incentive
structure resulting in the construction of widely available secure
systems, both U.S. and world businesses will continue to be largely
bare-ass naked where security is concerned. Further, it follows from
this that the U.S. civilian sector is vulnerable to attack in the event
of an extended military action. It has long been recognized that the
civilian sector is *vital* for provisioning and supply in any extended
military action (i.e. longer than one week).

In consequence, I would go so far as to argue that the current ITAR
policy is actively *contrary* to U.S. interests in both the commercial
and military sectors. Paul Karger has gone further, suggesting that "...
the current state of civilian sector computer security and the ITAR
regulations represents a clear and present danger to the security
interests of the United States."

For myself, I prefer a slighly less confrontational approach. My answer
generally goes something like: Well, we've been operating under the
current policy for 35 years, and we're pretty much naked. If you want
the civilian sector to be even modestly more secure, perhaps
criminalizing the open sale of secure systems isn't the way to go. All
other things being equal, it is a legitemate national security goal to
prevent our enemies from having this technology. The cost of the current
policy is that we are more vulnerable than they are, because we rely
utterly on computers in our civilian infrastructure, and most of our
enemies do not. The current policy is a knife at our own throats.


So finally, Adam was really asking "what do we do about it?" The key, I
think, is to recognize that international treaties don't matter. If a
reputable group of recognized computer scientists were to publish a well
thought out set of evaluation criteria for higher level evaluation, NSA
would have little political choice but to adopt them -- perhaps with
modification. Similarly, performing a successful evaluation against
these criteria using publicly contributed resources would disenfranchise
the people who say "real security is too hard". Both activities would
have to be done in a way that was very much beyond reproach. In part,
this means that it needs to be done in an open source way -- that is,
*all* of the process documentation and such needs to be publicly
reviewable.

Enough said for now, but I will hopefully have more to say on this
within the next few months.


shap


---------------------------------------------------------------------
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to majordomo at wasabisystems.com