[Cryptography] floating point

[Jerry Leichter]

On Dec 23, 2014, at 10:03 PM, John Denker <jsd at av8n.com> wrote:
>> I still remain astonished, that, since Babbage/Lovelace/Turing/etc, that 
>> many alleged programmers still do not know that 0.0 != 0.
>> 
>> Never compare floating-point numbers for equality; I think Henry Spencer 
>> said that (I've actually shook hands with him, and a better bloke you 
>> would never meet).
> I have no idea what that is trying to say, but whatever it is,
> that's not the right way to say it.
> 
> There are some things that you can do with floating point, and
> something you cannot.
> -- On any machine that uses IEEE floating point, in any normal 
>  language such as c or python, if one external representation is 
>  "0.0" and another is "0.", they have the same internal representation.
>  Exactly the same.  Exactly equal.  So count me as one of the "alleged
>  programmers" who thinks zero is equal to zero.
I think he was discussing comparing floating 0.0 with integer 0 - that last "." is the period at the end of the sentence,  Nevertheless, any reasonable hardware and software implementation that allows such a comparison (by converting both to a common representation) will decide they are equal.

In any areas, there's a general, easy to state rule that will keep you safe. "Don't compare floating point numbers for equality" is such a rule.  Aa you come to understand the field better, you learn that the general rule can be broken in many cases - e.g., "small enough" integers, exactly representable values.  The cases are complicated enough that we don't try to summarize them in a simple rule:  You need to be wary not just of the mathematical values but of the way they came to be computed.  Just because the mathematical value of an expression is an exactly representable number doesn't mean that the evaluation of that expression using FP arithmetic actually produces that value!

For most people who don't dig deeper, following the two rules "Don't compare floating values for equality" and "don't subtract floating values that are very close to each other" will avoid many of the common traps of floating point.  (Actually, there's another one that has no really simple statement, but "never print more digits of precision than your floating point numbers can actually represent" is one way of getting at it.  Following this rule avoids all kinds of traps people fall into when they think they can get more precision out of a calculation than the numbers themselves can actually carry.  Seen it happen....)

                                                        -- Jerry