# [Cryptography] Entropy is forever ...

John Denker jsd at av8n.com
Mon Apr 20 16:58:18 EDT 2015

```Clarification: The full, correct statement is:

For any given distribution,                        [a]
the entropy is a property of the distribution      [b]
... not of any particular string that may have
been drawn from such a distribution.               [c]

When people are in a hurry, common practice (but not good
practice) is to assert part [b] without being explicit about
part [a].  An example is my message from 04/17/2015 11:59 AM.

Beware that in part [b], the word «the» must not be given
undue emphasis, because it is heavily modified by part [a],
and must not be taken out of context.  That is, we are
*not* talking about «the» one true distribution.  In
cryptography and many other situations, it is common to
have more than one distribution in use at the same time.

Even if you know the string, you don't know «the» distribution
from which it was drawn.  In a card game, the microstate
(i.e. the objective state of the cards) is the same for
everybody, but different players see the probabilities
differently, i.e. they are working with different
distributions.  For further discussion including a
diagram, see
https://www.av8n.com/physics/thermo/entropy.html#sec-micro-macro

In crypto, it is virtually always the case that the sender
and the attacker are using wildly different distributions
over plaintexts and keys.  The microstate is known to the
sender, whereas the attacker presumably has to guess.  The
microstate is the same for both, but the distributions are
different.

Therefore it is madness to speak of «the» entropy inherent
in a string.  It is not possible to ascertain or even define
«the» one true distribution, not using Turing machines,
not using philosophy, not by any means whatsoever.

In the crypto business, more often than not, when people
distribution /as seen by the attacker/ ... but I am not
touting this as a reliable rule.  The smart approach is
to identify explicitly the distribution you are talking
about ... or (!) to carry the distribution as an unbound
variable.

Yes, I know one can find lots of references to the
idea of «the» «inherent» entropy.  By the same token,
-- Climate change is a hoax:
http://www.amazon.com/The-Greatest-Hoax-Conspiracy-Threatens/dp/1936488493
http://www.conservapedia.com/Global_warming
-- Dinosaurs coexisted with humans:
http://creationmuseum.org/whats-here/exhibits/allosaur/
-- The Apollo moon landings were fake:
http://www.imdb.com/title/tt0119654/quotes
-- et cetera..............

Similarly, I am quite aware that wikipedia says that
both energy and entropy are extensive variables:
http://en.wikipedia.org/wiki/Intensive_and_extensive_properties#Examples
However, it's just not true (not for energy or entropy),
especially for smallish systems, where surface states
make a nontrivial contribution.  I hate to belabor the
obvious, but wikipedia is not a reliable authority for
the foundations of physics or information theory.  It
doesn't try to be.  It explicitly doesn't want to be.

```