4

I just came across this piece of code:

Dim d As Double

For i = 1 To 10
  d = d + 0.1
Next

MsgBox(d)
MsgBox(d = 1)
MsgBox(1 - d)

Can anyone explain me the reason for that? Why d is set to 1?

1

3 Answers 3

8

Floating point types and integer types cannot be compared directly, as their binary representations are different.

The result of adding 0.1 ten times as a floating point type may well be a value that is close to 1, but not exactly.

When comparing floating point values, you need to use a minimum value by which the values can differ and still be considered the same value (this value is normally known as the epsilon). This value depends on the application.

I suggest reading What Every Computer Scientist Should Know About Floating-Point Arithmetic for an in-depth discussion.


As for comaring 1 to 1.0 - these are different types so will not compare to each other.

6
  • and why when I print it out its 1? and if so what might be the best way to compare it? thks!
    – Bartzilla
    Sep 16, 2011 at 19:44
  • @Eduardo - How do you print it out?
    – Oded
    Sep 16, 2011 at 19:45
  • I used MsgBox(d) or Console.WriteLine(d)
    – Bartzilla
    Sep 16, 2011 at 19:47
  • @Eduardo - That's because (1.0).ToString() returns "1".
    – Oded
    Sep 16, 2011 at 20:20
  • is .toString implicitly been called? when using MsgBox or Console.WriteLine?
    – Bartzilla
    Sep 16, 2011 at 20:22
3

.1 (1/10th) is a repeating fraction when converted to binary:

.0001100110011001100110011001100110011.....

It would be like trying to show 1/3 as a decimal: you just can't do it accurately.

0
2

This is because a double is always only an approximation of the value and not the exact value itself (like a floating point value). When you need an exact decimal value, instead use a Decimal.

Contrast with:

Dim d As Decimal

For i = 1 To 10
    d = d + 0.1
Next

MsgBox(1)
MsgBox(d = 1)
MsgBox(1 - d)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.