[Cryptography] Use Linux for its security

Henry Baker hbaker1 at pipeline.com
Fri Sep 30 16:24:38 EDT 2016

At 11:34 AM 9/30/2016, Ray Dillinger wrote:
>I would REALLY like to see a guarantee in the specification of fscanf
>and friends that "%s" when not length-qualified is considered equivalent
>to "%255s", or that using them to read more than 64k characters at a
>time is an error, or something, just to guarantee that there's a clearly
>"safe" amount of buffer to allocate.

I've had a long-standing complaint with Common Lisp -- even though it is
technically a buffer-overflow-free zone -- regarding arrays whose length
is a priori unknown.

The problem is that the Lisp "reader" doesn't know how long an array is
prior to reading elements from an input stream.  This means that a lot
of mechanism (and associated bugs & inefficiencies) is required to buffer
the incoming data in such a way that it eventually gets stored into a
correctly-sized array.  This also *guarantees* that Lisp can never compete
with some other languages in the efficiency of high volume I/O.

One obvious solution is to construct a Lisp list of elements until the
last element has been read, at which time the length is known and the
array can be allocated and filled.  But this is terribly inefficient,
since there are a lot more memory references than should strictly be
necessary.  (Note that in a garbage collected language, you need to
charge every allocation with a certain amount of amortized GC cost.)

If the size of the array were a priori known, then a correctly-sized
array could be *lazily* allocated and filled as the data came in.

By "lazy allocation", I mean something akin to OS virtual memory
page tables, where allocated (but not yet "touched") pages can be
assumed to be all zeros, and any page elements not written will
be subsequently read as zero.

Bottom line: my proposal for Common Lisp allowed for current
(Common Lisp standard) behavior with unknown array size, but
if that array size were provided in advance, then an alternative --
but far more efficient -- implementation would be invoked.

More information about the cryptography mailing list