Apologies for error, it's been a while since I did this :). I'll be using 8 bit numbers where relevant. I'm well aware that most systems use 32 and 64 bit, but this is _much_ simpler.

The problem is a simple one - representing fractions in binary.

As I'm sure most will be aware, a decimal integer, can be represented as a sum of powers of two.

Eg.

100 = 64 + 32 + 4 = 2 ^ 6  + 2 ^ 5 + 2 ^ 2
                  = 1100100
[download]

Positionally, the binary number is increasing powers of two. So the rightmost is 2 ^ 0, and the leftmost is 2 ^ 6.

The problem appears when we try and represent a fraction in binary.

In order to represent non-integer numbers, clearly we need a binary point (not decimal :)).

So we can have 1000.1010. The 'rule' for numbers to the right of the point, is that the 'powers' are negative. Eg. in decimal

10 ^ -1 = 0.1 
10 ^ -2 = 0.01
10 ^ -3 = 0.001
[download]

If we continue to work in binary, the same logic applies. Unfortunately:

2 ^ -1 = 1/2
2 ^ -2 = 1/4
2 ^ -3 = 1/8
binary 1010.1010 = 2 ^ 3 + 2 ^ 1 + 2 ^ -1 + 2 ^ -3 = decimal 10.625
[download]

As you can see, there are quite a few decimal fractions (eg. 0.1) that become recurring fractions when represented as a binary number.

This is the source of that particular quirk you noticed.

Floating point numbers are functionally similar to fixed point in this regard.

The weakness of fixed point becomes quickly obvious - if you 'reserve' 4 bits for the integer, and the other 4 for the mantissa then you end up with an 8 bit number being from 0 to 16 rather than the 0-256 range you'd have normally.

The workaround was to use floating point. Essentially, the 8 bit number gets cut up into 3 parts. The integer, the mantissa and the exponent. The exponent is a multiplier of 'powers of 2' of the binary number.

Although I'll leave it there, because there's many better explanations of floating point numbers on the net.

Essentially, the problem in your calculation comes from rounding errors when converting to/from binary fractions.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.