[Cryptography] A question on Shannon's entropy

[John Denker]

On 03/03/2016 09:50 AM, Ray Dillinger wrote:

> Because Shannon Entropy is not defined on
> sources unless they are producing independent outputs, this
> source cannot be said to have an entropy measure.

I wouldn't have said that.  A source with correlations
can have a perfectly well defined entropy.  You might
need a slightly more sophisticated formula for calculating
the entropy, but the concept is not open to question.

You can even calculate the entropy of a cat state,
i.e. an entangled quantum state.  That takes the
idea of "correlation" to a whole nother level.

  S = Trace ρ log ρ