Weird calculator glitch

brianleahy

brianleahy

Colonel Panic
I got an unexpected result from the OSX calculator, and I'd be intersted to hear if anyone else gets it:

Open calculator.
Add up the following numbers, in this order:
1.1
3
1
0.6
0.7

You'll get the correct answer, 6.4
Then do Cmd-C (copy). In calculator, it's not necessary (or possible) to select the numbers, just do a Cmd-C.
Now open a text editor (textedit, for instance) and paste it.

*I* got something different.
 
When you say 'different', I guess you mean 6.399999999999999!

If so, this is a problem with 'decimal' numbers. Once you introduce at least one decimal value (a number other than a whole number) into a list, the calculator goes into 'real number' mode. This holds and manipulates numbers in a completely different way than for whole numbers - it (essentially) is ready for complex decimal calculations! In a scientific/mathematical environment, a value such as 6.3999999 etc will simply be rounded to 6.4 without a second thought. But I agree, if you are putting in precise values, you should expect a precise answer!

Notice you do not get the same problem with numbers 6, 4 and 3. When added, copied and then pasted into a text editor, this shows 13 - as we would want.

I hope this helps.
 
I just think it's interesting that the calculator "screen" shows the correct answer of 6.4, but the pasteboard gets 6.399999...

Even so - this is ADDITION, with 1 decimal place - nothing fancy. I'd be better off adding 11, 30, 10, 6 and 7 and then moving the decimal place in my answer.

Also notice that if you add them up in the reverse order, it doesn't happen!
 
if
x=0.999999999...
then
10x = 9.99999999...
and
10x - 1x = 9.999999999.... -(0.9999999.....)
i.e.
9x = 9
and
x = 1
so

.99999....= 1

Since the addenda have only one significant digit, rounding off is almost required, but why doesn't calculator do that?
 
This is from http://www.macintouch.com from their Mac OS X Reader Report section:

Date: Tue, 8 Oct 2002 12:40:28 -0500
From: "Cyrus D. Cantrell"
Subject: Rounding errors in Jaguar's Calculator


Here's a response to john's post about rounding errors in Calculator in 10.2. The rounding error that john found is a generic feature of IEEE-754 floating-point arithmetic, which is supported in hardware in modern general-purpose microprocessors (such as the PowerPC). Actual hand calculators use base 10 (for the most part), not base 2, so they don't have the same rounding errors.

If you calculate 0.7 - 0.65 in IEEE-754 single precision, and write out 15 significant figures (even though only 7 are accurate:), you get 0.500000119209290E-01.

If you calculate 0.7 - 0.65 in IEEE-754 double precision, and write out 25 significant figures (even though only 16 are accurate:), you get 0.4999999999999993338661852D-01. This agrees with what john found (0.04999999999999993).

Conclusion: The Calculator in Jaguar uses the built-in hardware support for IEEE-754 double precision.

FWIW, the default for floating-point arithmetic in C is double precision.

Now that I know that the Jaguar Calculator uses IEEE-754 double precision (in Advanced mode), I'll use it to verify numbers from programs that I'm writing. :)
Click to expand...

and:

Re: More Calculator rounding errors

Scott Bayes
'MacInTouch Reader' wrote: "Last year MacIntouch had reports on the 10.2 Calculator having rounding errors due to the implementation of double precision. And everybody made fun of the errors in the calculator in Windows. But I recently upgraded all my machines to 10.3 (so I can't go back and check to see if this error is in the earlier versions), but Calculator v3.1 can't even do the following simple subtraction correctly.."

This appears to be due to finite binary floating point precision, and is an expected effect. Most calculations within modern computers use binary floating point representation, and are not exact, just as we know that 1/3 can't be represented exactly as a decimal number, but is .333... (repeating forever). Binary math is a bit different (e.g. 1/5 can't be represented exactly in binary, though it's exactly .2 in decimal), but the principle is the same.

This is all described in the IEEE-754 binary floating math spec (my copy is at work, so I don't have the exact title). Suffice it to say that with properly implemented IEEE-754 double precision, math can be expected to be accurate to about 16 decimal places (log(2^53) to be more precise) for any conversion from text such as "5195.27" to the computer's internal double precision form, and vise versa, back to text for calculation results. Your results are about that accurate, so it's probable there is no defect here. What may be occurring is that the previous version of the calculator was not displaying as many digits, so you didn't notice the effect of the approximation. My Panther is in a FedEx truck, probably somewhere in Nevada right now, so I can't say what Preference you might want to set in the calculator app to make the display look as you expect.


Tim Seufert
These are still just errors due to double precision floating point. To reiterate, computers use binary (base 2) number representation. Many fractions which can be exactly represented by a small number of digits to the right of the decimal point in base 10 require a large (or even infinite) number of digits to represent in binary. Since computers can only store a finite number of binary digits, the binary representations of such numbers are truncated, meaning the stored number is slightly different. "How bad do you have to be to not even get subtraction correct?" mischaracterizes what is going on -- it is not the subtraction that is at fault, it is the fact that some or all of the operands of the subtraction do not have an exact binary representation.

Obsessing about this is a waste of time.
Click to expand...

Apparently, calculators on computers (unlike hand calculators) use a different notation/base than we're used to, hence the rounding errors. Apparently, the display on the Calculator application uses one notation, while the paper tape uses another.
 
I'd be interested in knowing what processor everyone is using, just to see if perhaps it's hardware-specific. Mine's a G5..... ?
 
It's not hardware-specific... read the previous posts. It's the way the calculator makes its calculations.

And stop bragging. ;)
 
It's not a flaw, it's the way that the arithmetic is performed. And OS 9's calculator doesn't exhibit this "problem" because there's no paper tape function -- the paper tape in OS X displays the base-2 arithmetic, and also the copy function previously mentioned here uses base-2 arithmetic as well.
 
g3 10.3.2

9 strokes to make you laugh :)

but there is still the question of why it doesn't give you the repeating decimal if you add them up the other way.
 
Because the calculator "screen" uses base-10 to represent the number, while the paper tape and the copy function use base-2. That's why they show two different numbers, and that's why one is a repeating decimal and the other isn't.
 
Trust me, it's because of the base-2 and base-10 conversions. If you do the base-2 conversion yourself, you'll see that you get the same answer.

When you do it in reverse, one of the intermediate totals can be expressed as a whole number in base-2, so you don't get the repeating decimal.
 
I'd like that to make sense, but it just doesn't :P

In the first instance, the first decimal encountered is .1, which can be defined in binary. The repeating decimal is the .6, the second decimal in both the first instance and the second (adding in reverse order). In reverse order, the first decimal (.7) is an irrational repeating decimal (the 428571 repeats) and can't be expressed neatly in binary.

So the idea that an intermediate total (.7 the one way which is irrational and .3 the other way, another irrational) makes the final result different doesn't make sense (to me).

It is academic, no real problem, but my why still stands. (I don't doubt that it does make sense to you diablillo, it just doesn't to me yet.)
 
You must log in or register to reply here.
Back
Top