Forum Discussion

msaunders's avatar
msaunders
Contributor
11 years ago

Check point failing with matching values

Hi all, I have a checkpoint that is checking that the value is properly stored in an infragistics grid. it seems to fail at certain times through the test. I am stepping the value up by .1 at a...
  • HKosova's avatar
    11 years ago
    Hi Mathew,



    You're right, it happens because of floating point math specifics. The proper way to compare floating-point numbers is within some very small tolerance, like this:



    // Replace aqObject.CheckProperty with this:

    var diff = Math.abs(ultraGridCLT.wValue(1, "Scale") - i);

    if (diff < 0.000001)

      Log.Checkpoint("The values match.");

    else

      Log.Error("The values are different.");