[Cryptography] A question on Shannon's entropy

[Thierry Moreau]

On 02/03/16 11:31 PM, mok-kong shen wrote:
>
> Given 2 alphabetical sequences A and B of the same length. If A
> has higher entropy than B, then (if I don't err) this is commonly
> interpreted in crypto to imply that A is more random than B. Now
> suppose A has entropy u. Sort A to result in C such that all a's
> are in front of all b's etc. Since the frequencies of the characters
> are not affected by the sorting, C has also entropy u according to
> Shannon's formula. However C is evidently less random than A. How
> could one resolve this prolem?

Wrong application of Shannon lessons.

An entropy measure is tied to a data source that can emit codes without 
a size limit for the output.

In your question A and B are not data sources. If they were to be 
equated to a data source, the respective codes would have to be defined, 
e.g. as "alphabetic sequences of length-of-A" for A, and then sorting a 
sample from A creates a very different data source.

Said otherwise, the Shanon entropy of a sequence makes no sense. You 
must make assumptions of the data source behind the sequence and analyze 
the data source itself with the Shanon lessons.

- Thierry

  One could eventually employ pairs
> of characters as unit and compute a similar measure with the
> formula. But evidently the probllem would at best be weakened but
> not basically resolved that way.
>
> M. K. Shen
> _______________________________________________
> The cryptography mailing list
> cryptography at metzdowd.com
> http://www.metzdowd.com/mailman/listinfo/cryptography