[Cryptography] [Crypto-practicum] Justify the sequence of operations in CTR mode.

[Theodore Ts'o]

> Technology does advance, though.  When you were talking about
> spinning rust, it was essential that any metadata associated with a
> block be physically part of the block - putting something like an IV
> off elsewhere in a metadata area would destroyed performance.  With
> SSD's, the cost of an extra read is much less, and you might be able
> to get away with separated metadata.  On the other hand, you have to
> write in large pages, so updating that information is expensive.  So
> a mode that could rely on a *fixed* set of IV's might be a big win.

The biggest problem with separated metadata is the atomic update
problem.  What if you've updated the disk block, but not the
authentication/integrity metadata?  Or vice versa?

> New storage technologies are emerging, and what we know about good
> and bad designs for the disk stack is increasingly out of date.  As
> we develop new data structures appropriate to these new
> technologies, it would be a really good idea to consider
> cryptographic support from the ground up.

The challenge here is economic.  You can get disk drives that have
enough space for checksums; there is a SCSI standard for adding a Data
Integrity Field.  This was intended for end-to-end checksums, so the
disk will verify the checksum from the DIF field when the block is
read from spinning rust.  But one could imagine doing something very
much like this for crypto checksums where the disk drive would keep
its grubby little paws off of the authentication tag field.

The problem is $$$.  Drives with DIF/DIX are only available on
enterprise-grade SAS drives.  Which are so expensive that even the
major cloud companies tend not to use SAS drives, because they're
priced rapaciously.  If the HDD vendors had been smart enough to price
DIF/DIX as a only a tiny price premium then OS's would have started
using DIF/DIX, and then there would have been demand for such a
feature.  But without the demand, it's only a speciality / niche
feature, and it gets priced accordingly.

So not only do we have to consider cryptographic support from the
ground up, we have to figure ways of solving the demand side of the
equation.  I consider the lack of DIF/DIX in SATA grade drives to be a
market failure, and I can forsee the same sort of dynamic making
non-power-of-two storage devices as also a niche feature.  So short of
some kind of government mandate, or big-company mandate (we are
willing to purchase gazillion eMMC flash drives that have a block size
of 4160) I'm a bit pessimistic that this kind of redesign from the
ground up will happen.

Cheers,

					- Ted