27 May, 2009, Davion wrote in the 1st comment:
Votes: 0
I noticed something really weird today with floats, and different languages!

In python
In [1]: 664.79+1350.71+394.03+246.02
Out[1]: 2655.5499999999997


That isn't the right answer! So I decided to test it out with C++ to see if the result was any different.

me@skelly:~/test$ cat test.cpp
#include <iostream>

int main( void )
{ float i;

i = 664.79f + 1350.71f + 394.03f + 246.02f;
std::cout << i << std::endl;
return -1;
}

me@skelly:~/test$ ./test
2655.55


Got any ideas why python would output the wrong number, but C++ handles it fine?
27 May, 2009, David Haley wrote in the 2nd comment:
Votes: 0
It's possible that the C++ compiler is doing some kind of constant folding based on the input, or the output routine is doing some kind of rounding for you. It's not uncommon for some floating point operations to introduce tiny epsilon differences. This is generally fine unless you're trying to test literal equality; it's generally better to test epsilon-equality i.e. equality within a certain (very small) error margin epsilon.
27 May, 2009, Runter wrote in the 3rd comment:
Votes: 0
Not really sure what the beef is about here?


cout.precision(meh); // round to mehth

// or

cout << setprecision(meh) << i;
27 May, 2009, David Haley wrote in the 4th comment:
Votes: 0
So yes, it's in the output routines. (That's why I suggested the various things on IMC, to see if the "error" was being introduced before getting to the output.)
27 May, 2009, Tyche wrote in the 5th comment:
Votes: 0
28 May, 2009, David Haley wrote in the 6th comment:
Votes: 0
TLDR, is there a summary version…?
28 May, 2009, elanthis wrote in the 7th comment:
Votes: 0
You probably already know all the key bits, David. It's an exhaustive text on floats, their mathematical representation, common binary formats, properties, and so on. Written with enough detail for real Computer Scientist types while still being fairly readable to untrained/hobbyist programmers, too.
28 May, 2009, Tyche wrote in the 8th comment:
Votes: 0
A. Every FP calculation introduces an error.
B. FP operations are not associative nor distributive.
C. The order of calculation and intermediate representation is not necessarily the same nor accessible to those programming in high-level languages.
D. Neither is it even consistent within C system routines across compilers and machine architectures.

That said…
You may get the exact same answer to a single FP calculation, like 1.9 + 5.6, in most languages.
28 May, 2009, Runter wrote in the 9th comment:
Votes: 0
Runter said:
Not really sure what the beef is about here?


cout.precision(meh); // round to mehth

// or

cout << setprecision(meh) << i;


Oh sorry, I read the original post wrong. :)) Darn my old eyes.
28 May, 2009, David Haley wrote in the 10th comment:
Votes: 0
Oh, ok, thanks for the summaries. I skimmed through it and saw several equations on properties due to the binary representation so I assumed it would be talking about related things but was wondering if there were other things I was missing; the text is 94 pages long after all. :wink: One of these days I'll read the whole thing for completeness's sake.
29 May, 2009, flumpy wrote in the 11th comment:
Votes: 0
btw David your episilon-equality is not always going to be the best solution for testing floats are equal. From The article that Tyche posted:

Quote
Incidentally, some people think that the solution to such anomalies is never to compare floating-point numbers for equality, but instead to consider them equal if they are within some error bound E. This is hardly a cure-all because it raises as many questions as it answers. What should the value of E be? If x < 0 and y > 0 are within E, should they really be considered to be equal, even though they have different signs? Furthermore, the relation defined by this rule, a ~ b |a - b| < E, is not an equivalence relation because a ~ b and b ~ c does not imply that a ~ c.


EDIT:
(sorry David :))
29 May, 2009, David Haley wrote in the 12th comment:
Votes: 0
I'm assuming you're talking to me, not Davion, because he didn't bring up epsilon equality. Obviously it is not a perfect solution, but it's good enough the vast majority of the time for very many purposes. This solution has worked just fine at a multi-billion dollar hedge fund for about two decades, so I'm guessing it can't be all that terrible…

EDIT:
It's also worth considering that the definition of the "best solution" is highly dependent on what it is you're doing.
29 May, 2009, Runter wrote in the 13th comment:
Votes: 0
flumpy said:
btw David your episilon-equality is not always going to be the best solution for testing floats are equal. From The article that Tyche posted:

Quote
Incidentally, some people think that the solution to such anomalies is never to compare floating-point numbers for equality, but instead to consider them equal if they are within some error bound E. This is hardly a cure-all because it raises as many questions as it answers. What should the value of E be? If x < 0 and y > 0 are within E, should they really be considered to be equal, even though they have different signs? Furthermore, the relation defined by this rule, a ~ b |a - b| < E, is not an equivalence relation because a ~ b and b ~ c does not imply that a ~ c.


EDIT:
(sorry David :))


Technically correct is the best kind of correct.
29 May, 2009, flumpy wrote in the 14th comment:
Votes: 0
David Haley said:
I'm assuming you're talking to me, not Davion, because he didn't bring up epsilon equality. Obviously it is not a perfect solution, but it's good enough the vast majority of the time for very many purposes. This solution has worked just fine at a multi-billion dollar hedge fund for about two decades, so I'm guessing it can't be all that terrible…

EDIT:
It's also worth considering that the definition of the "best solution" is highly dependent on what it is you're doing.


cool but i don't think i'll be investing anytime soon tho ;)
29 May, 2009, David Haley wrote in the 15th comment:
Votes: 0
I'm going to assume for now that you're being facetious, Runter.
29 May, 2009, David Haley wrote in the 16th comment:
Votes: 0
flumpy said:
cool but i don't think i'll be investing anytime soon tho ;)

I wouldn't expect you to, for many reasons. Suffice it to say that we've done rather well for ourselves, even in the past few months.
29 May, 2009, quixadhal wrote in the 17th comment:
Votes: 0
Add to the confusion, by the fact that not all languages do the same thing when converting between types. :)

In C, (int) 4.5 == 4
In Perl, (int) 4.5 == 5

Perl uses rounding rules, C truncates.

In general, if you really want things to work as you expect, don't use floating point…. use fixed point. Unfortunately, not many languages have direct support for fixed point notation, so you either end up using a library with all the ugliness of using non-standard data types, or you do it yourself by using integers and doing the multiplication/division to shift things around. Neither solution is very good, IMO.

When dealing with money, I believe the quote was "Only people who don't know any better, or who are trying to get away with something, would use floating points."

DATA DIVISION.
WORKING-STORAGE SECTION.
01 MY-MONEY PIC T9(15)V99.
29 May, 2009, David Haley wrote in the 18th comment:
Votes: 0
Quote
When dealing with money, I believe the quote was "Only people who don't know any better, or who are trying to get away with something, would use floating points."

How many people who say things like this actually work in finance? I mean, all ya'all realize that this works just fine for the vast majority of purposes, right?
29 May, 2009, quixadhal wrote in the 19th comment:
Votes: 0
Of course, it's not MY quote, it was probably my COBOL instructor that I'm recalling. :)

People are paranoid about money, as I'm sure you know. They've seen too many movies where someone uses the roundoff errors to steal $0.0001 from everyone and fly off to the Bahamas. The places where it actually might matter are in scientific computing, where being off by a tiny fraction might cause you to go down the wrong research path, and cost millions.

I forget which math library does it, but my favorite solution to some of the problems was one that kept all results as rational numbers, not decimals. It would convert the results to decimal on demand (for output, usually), but by storing it as rationals, you can do things like X / 3, and not end up with artifacts due to 1/3 not being representable in a binary floating point system. That is also, probably, overkill for this venue. ;)
30 May, 2009, elanthis wrote in the 20th comment:
Votes: 0
Also fails to solve the problem of irrational numbers. :)

A lot of libraries exist which simply store numbers in a format with a guaranteed decimal precision. Some even can store numbers of arbitrary size, limited only by system memory. If you know that you need 16 digits of precision after the decimal point, that's pretty much the only way to deal with both rational and irrational numbers and get acceptable behavior from the system.
0.0/61