[Cryptography] Hierarchical bulk digest modes

[Darren Moffat]

You have basically described how the ZFS filesystem works for error
detection and correction.
It builds a Merkel hash tree of the per block hashes, stored in block
pointers containing metadata (compression, encryption, hash algorithm ids,
transaction birth id and information about the (upto) 3 copies of the
block).
In addition to RAID it also stores copies (always a minimum 2 copies of
metadata).
Block sizes are variable with 512 as the smallest and 1MB as the largest.
The hashes are also able to be used for inline deduplication.
When you have multiple copies you can not only detect but correct
corruption by reading from an alternative copy and then issuing a repairing
write.
ZFS is also copy on write, so it never overwrites live data.

When I added encryption to ZFS (in Solaris) I used AES in CCM or GCM. The
IV and tag are stored in the blockpointer.


Darren J Moffat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.metzdowd.com/pipermail/cryptography/attachments/20210830/b60ad730/attachment.htm>