[Cryptography] Cubbit

[Peter Fairbrother]

On 08/06/2020 19:09, Nemo wrote:
> On Sun, Jun 7, 2020 at 3:28 PM Tony Patti <crypto at glassblower.info 
> <mailto:crypto at glassblower.info>> wrote:
>  >
>  >
>  > > Jerry Leichter <leichter at lrw.com <mailto:leichter at lrw.com>> writes:
>  > >
>  > > > The description Cubbit provides says that they distribute 36 
> copies in
>  > > > a Reed-Solomon code that allows recovery from any 24 copies.  But one
>  > > > way or another ... 36 copies requires that, somewhere, there be 35
>  > > > times the space of the original copy to provide the redundancy.
>  > >
>  > >Nemo <nemo at self-evident.org <mailto:nemo at self-evident.org>> writes:
>  > >
>  > > We have a RAID6 at work that uses 12 1TB drives. It provides a full 
> 10TB
>  > of usable storage, and any two of those 12 drives can fail without 
> causing
>  > us any data loss whatsoever.
>  > >
>  > > How does this fit with your analysis?
>  > >
>  > >  - Nemo
>  >
>  > RAID5 and especially RAID6 have TERRIBLE write performance, which you can
>  > quantify here: https://wintelguy.com/raidperf.pl
> 
> RAIDx write performance is fine if you work with files in the hundreds 
> of gigabytes and write to them in large sequential chunks, which we do. 
> I will stop here before I dox myself.
> 
> The point is that 2x, 3x, 4x, etc. redundancy is quite possible without 
> doubling, tripling, or quadrupling your storage, contra Jerry's 
> analysis. I have not studied Cubbit, but it sounds like you can lose a 
> third of your 36 "partners" and still recover all of your data at a 
> storage increase of 1.5x.

Yup.

I don't know what this type of coding is properly called, but from your 
description it is not a standard Reed-Solomon coding.

Schneier refers to this sort of coding as a (m,n) threshold scheme 
(Applied Cryptography 2nd ed under s.3.7 secret sharing).

It has also been called erasure coding or block erasure coding.


One method is to create an m-dimensional array of some data then take n 
shadow snapshots of it. Each snapshot (shadow) is approximately N/m in 
size (strictly, < N/(m-1) ). Any m shadows can recreate the original.

See: G.R. Blakley, “Safeguarding Cryptographic Keys,” Proceedings of the 
National Computer Conference, 1979, American Federation of Information 
Processing Societies, v. 48, 1979, pp. 313–317.


If you are not trying to be cryptographically secure (as in Cubbit) then 
there are quicker methods, especially for small m and n.

These can be very much faster than RAID codings, but they do not detect 
or correct bit-level errors.

Peter Fairbrother