# [Cryptography] Entropy is forever ...

John Denker jsd at av8n.com
Sat Apr 25 01:07:45 EDT 2015

```On 04/24/2015 04:12 PM, Bill Cox wrote:

> We can't fix how everyone else uses the word entropy.

OTOH we ought not run around making the problem worse.

> Linux has an entropy pool...

That's true.

>  end of story.

Absolutely not end of story, for multiple reasons:

*) We can quite reasonably discuss the fact that the
entropy is not "in" the pool, but rather in the ensemble
of similarly-prepared pools.
*) We can quite reasonably ask whether the code keeps
track of the entropy correctly.
*) We can discuss the fact that the driver is completely
at the mercy of upstream entropy providers. It depends
on them to provide entropy, and to correctly say
/how much/ entropy is being provided.
*) We can also discuss the fact that what the code
actually /does/ is different from what the comments
say it does ... and what the symbol-names imply it
does.
*) We can also quite reasonably ask whether entropy
is the right thing to be keeping track of.

>   we can be more clear in our discussions of
> TRNGs, randomness, and entropy.  In common usage, it is simpler to say
> entropy, because no one knows what you're talking about if you say
> surprisal :-)

You could define your terms as you go along.

If you say entropy when you mean surprisal, it may be easier
for you to say, but it's eeeeeenormously harder for everybody
else to figure out what you're talking about.  If you want to
be understood, it helps to use the terminology correctly.

On 04/21/2015 09:54 PM, dj at deadhat.com wrote:

>> I googled the term [surprisal] and it seem to be used in thermodynamics circles.

Yes, there ... along with lots of other places.  The term
"surprise value" has been in the communication-theory
literature for well over 50 years that I know of, possibly
longer/1/.

>> The
>> math definition doesn't seem to provide any cause to abandon Renye Entropy
>> as the normal metric.

"The" normal metric?  I think that's overstating it.

For starters, here's a whole family of Rényi functionals.  If
you just say "Rényi" without qualification, people will assume
H[2], which has to do with collisions.  It might be relevant
or it might not.  If you mean H[1] that's just the plain old
Shannon entropy.  If you mean H[∞] you should please say so.
It has some relevance directly, and also indirectly insofar
as it provides a lower bound on the plain old entropy.

Secondly, it depends on what you're doing.  As usual, we
What's Your Use Case.  It also depends on whether we are
optimizing things from the message-sender's point of view
or the attacker's point of view.

Surely there are /some/ quite-reasonable threat models where
we care about the min-entropy H[∞].  I'm 100% supportive right
up to the point where it is claimed to be "the normal metric".
/Sometimes/ it measures something important.  Sometimes it is
necessary and sufficient, but sometimes it is merely sufficient.

In the parts of the world where I live, there is rather little
one can do with Shannon entropy alone, or with algorithms alone.
The combination of the two is a lot more interesting.  The same
goes for min-entropy plus algorithms:  The combination is more
useful than either one alone.  In particular, the min-entropy
might tell us that one of the output codes is at risk of being
enormously over-represented.  That doesn't matter, if the
attacker has to wait 10^10 times the age of the universe before
seeing that code.

A large hash buffer size ("pool" size) covers a multitude of sins.

/1/ C. Cherry _On Human Communication_ (1957) page 179 among others.

```