4

Question: How does the JVM compute the sum of two double-precision NaNs?

Details: The IEEE 754 specification reserves two ranges of bit patterns for NaNs:

0x7ff0000000000001 -> 0x7fffffffffffffff

and

0xfff0000000000001 -> 0xffffffffffffffff.

IEEE 754 requires the sum of two NaNs to be a NaN, but, as far as I can tell, is silent on implementation details. So, to get to my question: If we write b(x) for the hexadecimal bit pattern of a NaN x, I'd like to know: How does the JVM calculate b(x + y) from b(x) and b(y)? Playing with a little bit of code, I'm lead to believe:

Claim: Let t = 0x0008000000000000. If b(y) + t is in a valid NaN range, then:

b(x + y) = b(y) + t

otherwise,

b(x + y) = b(y).

This seems strange to me, and I'd like to know more. For the record, I'm using Java 8 on an Intel i7 MacBook (in case the Java version or physical hardware matter.) Here are two examples:

Example 1, where b(x + y) = b(y):

b(x) = 0x7fffddee0f43e7d4
b(y) = 0xfffaeaba08397e4e
b(x + y) = 0xfffaeaba08397e4e

Example 2, where b(x + y) = b(y) + t:

b(x) = 0xffff4f0202031106
b(y) = 0xfff79342c97104ff
b(x + y) = 0xffff9342c97104ff

Does anyone know how the sum is being evaluated by the JVM?

Thanks!

4
  • The JVM doesn't do this: the hardware FPU does, and it does so in accordance with the IEEE-754 rules.
    – user207421
    Mar 26, 2020 at 5:45
  • 1
    Note that the JVM actually uses the FPU and unless you use strictfp the FPU has greater precision than IEEE-754 (unless you run it on an IBM iSeries or Mainframe; that is a challenging "bug" to identify and correct). Mar 26, 2020 at 5:51
  • Commonly, the high bit of the significand indicates whether a NaN is signaling (0) or quiet (1). A common behavior for handling NaNs is that, when an operation has NaN inputs and should produce a NaN output, one of the input NaNs is used (since this helps trace the history of a NaN found later in the program). Additionally, it is changed to a quiet NaN. This matches the results shown in the question: The bits come from one of the operands and the signaling/quiet bit is set. Mar 26, 2020 at 12:01
  • Thanks for the comments, very helpful! @EricPostpischil your note about the significand was particularly illuminating. Mar 26, 2020 at 18:45

1 Answer 1

4

For floating-point, most implementations simply rely on what the underlying hardware does. Assuming you're running this on some sort of an x86 architecture, here're the rules: (You can find this in Section 4.8.3.5 of https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf):

enter image description here

1
  • Wonderful answer! Thanks so much, @alias! Mar 26, 2020 at 18:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.