[Cryptography] information, Shannon, and quantum mechanics

[John Denker]

On 02/23/2015 01:04 PM, Henry Baker wrote:

> Shannon was perhaps the last Victorian scientist

I have no idea what that's trying to say.  Queen V.
died 15 years before Shannon was born.  He was not
a steampunk, not even remotely.  Bell Labs in the
1940s and 1950s was not known for doing 1800s-stye
research.

> his
> information theory may well be at odds with quantum mechanics.

Shannon's ideas of information and entropy have long
since been reconciled with quantum mechanics.
  Shannon:     1948
  von Neumann: 1955

The fully-general expression is known.  Reference:
  http://www.theory.caltech.edu/people/preskill/ph229/notes/chap5.pdf

However, the answer is not very widely known.  Perhaps
that is as it should be, since Von Neumann's formula 
reduces to Shannon's formula in the correspondence 
limit.  The classical version is good enough for all 
applications you are likely to encounter, unless you 
go to a lot of trouble to set up a non-classical 
situation ... or you are answering trick questions.

  There is no doubt that such questions exist;
  see puzzle below.

> We still don't know what a bit _weighs_; 

Actually, Kemosabe, we do know.  The short answer is
zero.  In more detail:  there is no positive lower 
bound on the mass of a bit, or on the energy.  If 
you use a sufficiently sparse N-way coding, you can 
represent log(N) bits per photon (or per anything 
else).  This brute-force encoding suffices to prove
the point.

> resolving this (and other
> associated questions) will keep physicists occupied for perhaps the
> next 50 years.

Those are questions from 50 or 60 years ago.  We have 
lots of other questions to keep us busy now, and 50 
years from now.

=====================================
Here's a seemingly-simple puzzle:

Suppose we have a system that can be in either of two
microstates.  Actually we have an ensemble of such
systems, and repeated measurements have observed the
system to be in microstate "A" with probability 1/3rd
and microstate "B" with probability 2/3rds.

Two questions:

  1) What is the entropy of the system?

  2) How sure are you that your answer is correct?

Hint:  It's easy to get the right answer, but it's also easy
to get the wrong answer, depending on how you approach the 
problem.  Lots of "authoritative" references will lead you 
to the wrong answer.

The usual jsd puzzle rules apply:  Everything I've said here
is true and helpful, to the best of my knowledge.  However,
obviously I haven't told you everything;  notably, I haven't 
told you the answer.  This isn't a word game;  solving the 
puzzle requires understanding the physics, not quibbling 
about words.