0

I'm trying to detect whether I've multiplied the numerator and denominator of a complex fraction enough times (by 10/10) to get integers on both top and bottom. Often my algorithm works. Sometimes, though, the Int() function doesn't work as expected and I get huge numbers out of it. Here is a simplified version of what I'm running:

<cfoutput>
    <cfset RealNumber = RandRange(1000, 10000) / 1000 />
    RealNumber = "#RealNumber#" Int(RealNumber) = "#Int(RealNumber)#"<br />
    <cfloop condition="RealNumber NEQ Int(RealNumber)">
        <cfset RealNumber = RealNumber * 10 />
        RealNumber = "#RealNumber#" Int(RealNumber) = "#Int(RealNumber)#"<br />
    </cfloop>
</cfoutput>

Most of the time this gets what I want, but sometimes it looks like this:

RealNumber = "9.184" Int(RealNumber) = "9"
RealNumber = "91.84" Int(RealNumber) = "91"
RealNumber = "918.4" Int(RealNumber) = "918"
RealNumber = "9184" Int(RealNumber) = "9183"
RealNumber = "91840" Int(RealNumber) = "91839"
RealNumber = "918400" Int(RealNumber) = "918399"
RealNumber = "9184000" Int(RealNumber) = "9183999"
RealNumber = "91840000" Int(RealNumber) = "91839999"
RealNumber = "918400000" Int(RealNumber) = "918399999"
RealNumber = "9184000000" Int(RealNumber) = "9183999999"
RealNumber = "91840000000" Int(RealNumber) = "91839999999"
RealNumber = "918400000000" Int(RealNumber) = "918399999999"
RealNumber = "9.184E+012" Int(RealNumber) = "9.184E+012"
RealNumber = "9.184E+013" Int(RealNumber) = "9.184E+013"
RealNumber = "9.184E+014" Int(RealNumber) = "9.184E+014"
RealNumber = "9.184E+015" Int(RealNumber) = "9.184E+015"

Or even like this:

RealNumber = "2.152" Int(RealNumber) = "2"
RealNumber = "21.52" Int(RealNumber) = "21"
RealNumber = "215.2" Int(RealNumber) = "215"
RealNumber = "2152" Int(RealNumber) = "2152"
RealNumber = "21520" Int(RealNumber) = "21520"
RealNumber = "215200" Int(RealNumber) = "215200"
RealNumber = "2152000" Int(RealNumber) = "2152000"
RealNumber = "21520000" Int(RealNumber) = "21520000"
RealNumber = "215200000" Int(RealNumber) = "215200000"
RealNumber = "2152000000" Int(RealNumber) = "2152000000"
RealNumber = "21520000000" Int(RealNumber) = "21520000000"
RealNumber = "215200000000" Int(RealNumber) = "215200000000"
RealNumber = "2.152E+012" Int(RealNumber) = "2.152E+012"
RealNumber = "2.152E+013" Int(RealNumber) = "2.152E+013"
RealNumber = "2.152E+014" Int(RealNumber) = "2.152E+014"
RealNumber = "2.152E+015" Int(RealNumber) = "2.152E+015"
RealNumber = "2.152E+016" Int(RealNumber) = "2.152E+016"

What am I doing wrong and how do I compensate for this?

3 Answers 3

2

RealNumber = "9.184E+015"

The output is deceptive. If you use #RealNumber.toString()# the real value is probably something like 9.183999999999998E15. CF uses the approximate type Double for most mathematical operations. So what you are seeing is normal behavior for floating point numbers. If you need greater accuracy, use PrecisionEvaluate. It makes use of BigDecimals for arithmetic operations (because they are more precise than java.lang.Double).

2
  • <cfloop condition="RealNumber NEQ PrecisionEvaluate(Int(RealNumber))"> still gets the huge numbers. Feb 7, 2012 at 13:31
  • That is because you have to use it on both values ie precisionEvaluate(RealNumber * 10). Otherwise CF falls back to using floating point numbers when you perform math operations.
    – Leigh
    Feb 7, 2012 at 17:36
0

Try using javacast.. ColdFusion is trying to dynamically determine your variable type and is not doing a great job.. Javacast covers a multitude of sins.

<cfoutput>
<cfset RealNumber = RandRange(1000, 10000) / 1000 />
RealNumber = "#RealNumber#" Int(RealNumber) = "#Int(RealNumber)#"<br />
<cfloop condition="RealNumber NEQ Int(RealNumber)">
    <cfset RealNumber = RealNumber * 10 />
    RealNumber = "#RealNumber#" Int(RealNumber) = "#javaCast("int", RealNumber)#"<br />
</cfloop>

1
  • This: <cfloop condition="RealNumber NEQ JavaCast('int', RealNumber)"> still gets the huge numbers. Feb 7, 2012 at 13:29
-1

Val() seems to correct it too:

<cfoutput>

    <cfset RealNumber = RandRange(1000, 10000) / 1000 />
    "#val(RealNumber)#" = INT: "#val(Int(RealNumber))#"<br />
    <cfloop condition="val(RealNumber) NEQ Val(Int(RealNumber))">
        <cfset RealNumber = RealNumber * 10 />
        "#val(RealNumber)#" INT: "#val(Int(RealNumber))#"<br />
    </cfloop>

</cfoutput>
2
  • <cfloop condition="Val(RealNumber) NEQ Val(Int(RealNumber))"> solves the problem and gets the least amount of loop time. Feb 7, 2012 at 13:32
  • @Dave - I am a little skeptical about that ;) PrecisionEvaluate usually results in far fewer iterations because it is not using approximate numbers like 9.183999999. So evaluating 9.184 requires four (4) iterations instead of ~thirteen (13) with val. That said, it is not a significant difference overall, unless you are handling hundreds or thousands of numbers.
    – Leigh
    Feb 7, 2012 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.