[Cryptography] [cryptography] Is it time for a revolution to replace TLS?

[Jerry Leichter]

On May 17, 2014, at 5:16 AM, ianG <iang at iang.org> wrote:
> Yes, this is the way in.  One problem that occurred with TLS was that
> there was an assumption that the job was to secure the reliable stream
> connection mechanics of TCP.  False assumption.
> 
> Pretty much nobody uses streams by design, they use datagrams.  And they
> use them in a particular fashion:  request-response.  Where we went
> wrong with TCP was that this was the easiest way to handle the mechanics
> of getting the response back to the agent that sent the request.
> Without TCP, one had to deal with the raw incoming datagrams and
> allocate them to the different sending agents.
This is an interesting assertion, much broader than cryptography but with a fundamental effect on it.

Prior to Unix and C, OS's and programming languages used a record model for I/O.  Historically, this undoubtedly arose in the US because the fundamental medium for I/O was initially punched cards.  Tape and disk formats emulated cards, and everything followed from there.  (You might think that fixed-length blocks on disks were another primary determinant, but in fact very early disks supported variable-length blocks, so probably were not as big a determinant.  Of course, tape drives also supported variable-length blocks.  In both cases, the general approach was to use a fixed maximum block length, and then "optimize" short blocks by using the variable-length mechanism to only write as much data as needed.  In Europe, cards were less common than paper tape, so some early machines and languages didn't have a strong block I/O bias; but the US completely dominated the industry in those days.)  The big advantage to record orientation was that you knew up front how large a buffer you needed to allocate.  Typical systems of the day did I/O directly into and out of user-provided buffers, for performance as well as total memory usage reasons.  Records with a fixed size, or a fairly small maximum size, were a good match.

Unix and C introduced the pure-stream-of-bytes model for I/O.  Unix got around the buffer problem by using an in-OS buffer pool, hiding the underlying fixed block length structure of disks.  They could get away with this because memory was getting cheaper and CPU's were getting faster - and the early Unix users weren't writing the kind of I/O intensive code that business users cared about (updated customer records:  maximum data access, minimum computation).

Early Unix fans were quite rabid about this being the only good model; forget all that old record-oriented stuff.  Besides, the argument went, if you want records you can always build them on top of streams.  I used to point out that this was false in an important sense.  FORTRAN was quite explicit in requiring that after the following sequence (ignore the syntax):

A = 1
B = 2
C = 3
WRITE A, B
WRITE C
...
READ X
READ Y

X will contain 1 and Y will contain 3 - the extra value of 2 in "the first record" is skipped.  It's impossible to design a file layout that simultaneously (a) allows a FORTRAN reader to get its desired semantics; (b) allows a "stream" reader to see 1 2 3 with no extra information.

Conversely, emulating a pure stream access method on top of a record-oriented one wasn't really possible either - as VMS demonstrated when VAX C was introduced.  Its library did the best it could, but significant rough edges (particularly around how fseek/ftell were supposed to seek into the "middle" of a record; but there were other issues) remained until VMS eventually added a native stream access method - in the process inevitably losing a certain level of universality of its files.  I'm not going to argue that the record-oriented system was better - making *terminals* look record-oriented (as VMS did) lead to some real absurdities.

Pre-TCP network protocols were designed using the familiar block-I/O principles.  Early systems didn't actually integrate their network I/O into their file systems - the concept of device-independence pretty much didn't exist at the time.  But people stick to tried-and-true principles and reuse designs.  TCP, of course, did the same on the flip side:  They designers took the Unix stream access method and applied it to networks.  (That Unix wanted to fit network connections into its general file I/O system only encouraged this design.)

Some early uses of TCP - FTP transfer, especially of stream-oriented files - was a perfect match.  (FTP and record-oriented file systems were always a problematic pairing.)  Others - email - fit with minor effort, adding a notion of "lines" - records in another guise - defined by terminators for certain purposes.  Telnet wasn't a great match - terminals as used by interactive programs aren't a great match to either record I/O or stream I/O and the delays of a remote connection made the mismatch very clear.  Many other programs found that they really wanted to exchange messages, which were essentially records.  They each got to build that for themselves, introducing generations of buffer overflow issues that plague us literally to this day.

SSL came at this and built a message (record) interface on top of TCP (because that was convenient for defining a crypto layer), and then a (mainly) stream interface on top of its message interface - because programmers were by now familiar with streams, not records.

And so ... here we are.  Living in a city built on top of generations of older cities.  Dig down and see the accreted layers.

What *is* the "right" (easiest to use correctly, hardest to use incorrectly, with good performance, across a large number of distinct application API's) underlying interface for a secure network link?  The fact that the first thing pretty much all API's do is create a message structure on top of TCP makes it clear that "pure stream" isn't it.  Record-oriented designs derived from 80-column punch cards are unlikely to be the answer either.  What a "clean slate" interface would look like is an interesting question, and perhaps it's finally time to explore it.
                                                        -- Jerry