Linked Questions

826 votes
47 answers
683k views

How to deal with floating point number precision in JavaScript? [duplicate]

I have the following dummy test script: function test() { var x = 0.1 * 0.2; document.write(x); } test(); This will print the result 0.020000000000000004 while it should just print ...
Juri's user avatar
  • 32.5k
6 votes
2 answers
21k views

Python floating-point math is wrong [duplicate]

Python 2.7.3 (v2.7.3:70274d53c1dd, Apr 9 2012, 20:52:43) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "copyright", "credits" or "license()" for more ...
gadgetmo's user avatar
  • 3,058
96 votes
8 answers
60k views

PHP - Floating Number Precision [duplicate]

$a = '35'; $b = '-34.99'; echo ($a + $b); Results in 0.009999999999998 What is up with that? I wondered why my program kept reporting odd results. Why doesn't PHP return the expected 0.01?
dcmoody's user avatar
  • 1,295
56 votes
6 answers
25k views

Is double Multiplication Broken in .NET? [duplicate]

If I execute the following expression in C#: double i = 10*0.69; i is: 6.8999999999999995. Why? I understand numbers such as 1/3 can be hard to represent in binary as it has infinite recurring ...
Dan's user avatar
  • 29.1k
98 votes
2 answers
154k views

Float sum with javascript [duplicate]

Possible Duplicate: Is JavaScript's Math broken? I'm calculating the sum of several float values using javascript and... I've noticed a strange thing never seen before. Executing this code: ...
davioooh's user avatar
  • 23.9k
86 votes
4 answers
149k views

JavaScript adding decimal numbers issue [duplicate]

So I am making a script that adds two numbers (decimal numbers) together, which I have encountered a problem. http://jsfiddle.net/DerekL/esqnC/ I made the script, it turns out pretty good: 0.1 + 0....
Derek 朕會功夫's user avatar
82 votes
5 answers
72k views

Why does adding two decimals in Javascript produce a wrong result? [duplicate]

Possible Duplicate: Is JavaScript’s Math broken? Why does JS screw up this simple math? console.log(.1 + .2) // 0.3000000000000004 console.log(.3 + .6) // 0.8999999999999999 ...
Greg Perham's user avatar
  • 1,855
57 votes
3 answers
80k views

Python modulo on floats [duplicate]

Can anyone explain how the modulo operator works in Python? I cannot understand why 3.5 % 0.1 = 0.1.
beruic's user avatar
  • 5,557
37 votes
7 answers
36k views

Floating point arithmetic not producing exact results [duplicate]

I need to do some floating point arithmetic in Java as shown in the code below: public class TestMain { private static Map<Integer, Double> ccc = new HashMap<Integer, Double>() { ...
user avatar
13 votes
6 answers
32k views

C++ How to avoid floating-point arithmetic error [duplicate]

I am writing a loop that increments with a float, but I have come across a floating-point arithmetic issue illustrated in the following example: for(float value = -2.0; value <= 2.0; value += 0.2) ...
Barney's user avatar
  • 2,365
30 votes
2 answers
52k views

Python rounding error with float numbers [duplicate]

I don't know if this is an obvious bug, but while running a Python script for varying the parameters of a simulation, I realized the results with delta = 0.29 and delta = 0.58 were missing. On ...
jpjandrade's user avatar
7 votes
10 answers
3k views

1.265 * 10000 = 126499.99999999999? [duplicate]

When I multiply 1.265 by 10000 , I get 126499.99999999999 when using Javascript. Why is this so?
vladaruz's user avatar
  • 229
25 votes
6 answers
19k views

Simple subtraction bug in Javascript [duplicate]

I was shocked to see such a basic subtraction bug in JS, I am sure most of you here would have experienced this issue, please help me with a work around here, All I am doing is subtracting a number ...
Umair Jabbar's user avatar
  • 3,596
25 votes
3 answers
15k views

Why does Math.cos(90 * Math.PI/180) yield 6.123031769111... and not zero? [duplicate]

I convert degrees to radians (degrees * Math.PI/180) but why does the following: Math.cos(90 * Math.PI/180) yield 6.123031769111... and not zero? I'm trying to perform 2D rotations uses matrixes ...
Sean's user avatar
  • 537
21 votes
2 answers
4k views

float/double Math.Round in C# [duplicate]

float ff = (float)31.15; double dd = 31.15; var frst = Math.Round(ff, 1, MidpointRounding.AwayFromZero); var drst = Math.Round(dd, 1, MidpointRounding.AwayFromZero); frst: 31.1 drst: 31.2 Can ...
Yaoqing's user avatar
  • 283

15 30 50 per page
1
2 3 4 5
329