Apologies for error, it's been a while since I did this :). I'll be using 8 bit numbers where relevant. I'm well aware that most systems use 32 and 64 bit, but this is _much_ simpler.

The problem is a simple one - representing fractions in binary.

As I'm sure most will be aware, a decimal integer, can be represented as a sum of powers of two.

Eg.

100 = 64 + 32 + 4 = 2 ^ 6  + 2 ^ 5 + 2 ^ 2
                  = 1100100
[download]

Positionally, the binary number is increasing powers of two. So the rightmost is 2 ^ 0, and the leftmost is 2 ^ 6.

The problem appears when we try and represent a fraction in binary.

In order to represent non-integer numbers, clearly we need a binary point (not decimal :)).

So we can have 1000.1010. The 'rule' for numbers to the right of the point, is that the 'powers' are negative. Eg. in decimal

10 ^ -1 = 0.1 
10 ^ -2 = 0.01
10 ^ -3 = 0.001
[download]

If we continue to work in binary, the same logic applies. Unfortunately:

2 ^ -1 = 1/2
2 ^ -2 = 1/4
2 ^ -3 = 1/8
binary 1010.1010 = 2 ^ 3 + 2 ^ 1 + 2 ^ -1 + 2 ^ -3 = decimal 10.625
[download]

As you can see, there are quite a few decimal fractions (eg. 0.1) that become recurring fractions when represented as a binary number.

This is the source of that particular quirk you noticed.

Floating point numbers are functionally similar to fixed point in this regard.

The weakness of fixed point becomes quickly obvious - if you 'reserve' 4 bits for the integer, and the other 4 for the mantissa then you end up with an 8 bit number being from 0 to 16 rather than the 0-256 range you'd have normally.

The workaround was to use floating point. Essentially, the 8 bit number gets cut up into 3 parts. The integer, the mantissa and the exponent. The exponent is a multiplier of 'powers of 2' of the binary number.

Although I'll leave it there, because there's many better explanations of floating point numbers on the net.

Essentially, the problem in your calculation comes from rounding errors when converting to/from binary fractions.

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.