[Cryptography] Is ASN.1 still the thing?

[Jerry Leichter]

> One of the reasons we have far too many serialization protocols is that we don't need a serialization protocol at all.  We don't want to serialize arbitrary data, because the whole point of serializing data is so that someone else can deserialize that data.  And he does not want to deserialize arbitrary data.  He wants to deserialize particular message types known in advance.
> 
> So, a serialization protocol needs to be part of, and an afterthought to, a message type negotiation that occurs when forming a connection, the sockets and protocol negotiation when forming a connection.  The two ends need to agree on a set of known message types that cannot represent arbitrary data, but can only represent a small set of object types known in advance to both ends at compile time.
One can easily find arguments going exactly the other way.  I happened to be looking at a book from a couple of years back called Bitter Java today.  It's in the "Antipatterns in XXX" vein, though not in that series.  An antipattern it describes is exactly what you recommend:  Using a transfer mechanism where things are "too tightly defined".  The example they give has a zip code field that's exactly 5 digits long (which breaks with the introduction of 9-digit zip codes); and of a "country code" represented as an enumerated type, which breaks when a new country comes into existence.  Changing these kinds of things is extremely difficult in a system composed of many components that evolve separately, often on very different schedules; and especially components owned by different authorities.

The fix they recommend is to use XML, because of its flexibility.  (Today, the recommendation would probably be JSON - mentioned in passing in the book, but fashions change.)

Of course, just exactly what a client that receives a country code it doesn't recognize is actually supposed to *do* about it is an interesting question.

There are some very difficult issues here.  We tend to love one-size-fits-all solutions, but sometimes this gives us solutions that fit all badly.

At one extreme, we have cryptographic protocols, whose bit representation need to be very tightly constrained, which need to leave no free play (unspecified bits), and which on the one hand need to be rigorously checked by the receiver; but on the other hand, that receiver had better not send back detailed error reports for fear of acting as an oracle.

At the other extreme, we have Internet protocols designed for large numbers of players to use, which are expected to change frequently in unpredictable ways, whose messages often should not be checked *too* closely by receivers for fear of failing to support "important" senders that "get some silly technicality wrong", and which should be very verbose and descriptive when they *do* report an error in the hopes that the sender can figure out what it needs to change.

And ... there are tons of things in between.

One of the things I find ironic in this whole discussion is the status of ASN.1 and its encodings.  On the one hand, ASN.1 is the product of an era when we thought rigorous definition of protocols was a necessity so that independent implementations could reliably communicate.  On the other, ASN.1 was also intended to support emerging styles of networked communications.  It's extremely general because of the perceived need to support all kinds of applications.  But ... no one at the time anticipated - or *could have* anticipated - what the Internet, the Web, Web services, and that whole ecosystem would really look like, and what the emergent properties of systems of that scale and organic growth, rather than explicit design and construction, would be.

So ASN.1 - and other designs of that era, like CORBA - end up in an uncomfortable middle ground:  They would be great for tightly-coupled, small-to-medium-sized systems that implement individual services in parallel and distributed pieces; but are more general than needed, hence appear way too complex.  Meanwhile they are a poor match to the very-large-scale, open-ended, organic "systems" of the greater Web.  (In the end, even XML faced the same problem for these systems:  Too complex to justify the benefits it claimed to offer.  Hence the victory of JSON.)

It's an old saw that every OS problem (and solution) was already present in Multics.  One could probably argue the similarly that every distributed system problem (and solution) was already present in CORBA.  And ... we're seeing explicit arguments on this thread that every problem (and solution) in serialization was already dealt with in ASN.1 and one or another of the xER's spawned to represent the data it describes.

But ... while historically fascinating and instructive, Multics isn't coming back.  CORBA isn't coming back, though it continues to be used in a some specialized, existing applications.  And ASN.1 and the xER's aren't coming back either.

                                                        -- Jerry