Equality Comparison of Floating-Point Numbers in C# (and Others)

tl;dr

You should compare two floating-point numbers for equality in C# as follows:

bool Equals(double x, double y, double tolerance)
{
    var diff = Math.Abs(x - y);
    return diff <= tolerance ||
           diff <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
}

tolerance must be much larger than double.Epsilon.

The problem in the equality operator

You should avoid using the equality operator == to compare two floating-point numbers for equality. This is because the equality operator can return false for two intuitively equal numbers. The following list and the output indicate the problem.

ar one = 0d;
for (var i = 0; i < 10; i++)
    one += 0.1;
var pairs = new[]
{
    (1.0 - 0.9, 0.1),
    (0.15 + 0.15, 0.1 + 0.2),
    (one, 1.0)
};
foreach (var p in pairs)
{
    var (x, y) = p;
    Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
        $"x == y: {x == y}, Abs(x-y): {Math.Abs(x - y):f18}");
}

x: 0.099999999999999978, y: 0.100000000000000006,
    x == y: False, Abs(x-y): 0.000000000000000028
x: 0.299999999999999989, y: 0.300000000000000044,
    x == y: False, Abs(x-y): 0.000000000000000056
x: 0.999999999999999889, y: 1.000000000000000000,
    x == y: False, Abs(x-y): 0.000000000000000111

The equality comparisons between 1.0 - 0.9 and 0.1, between 0.15 + 0.15 and 0.1 + 0.2, and the sum of ten 0.1 and 1.0 unexpectedly result in false. The errors in the floating-point numbers cause these results.

Tolerating absolute errors

The well-known way to resolve this problem is to compare the absolute error of two numbers with a minuscule number. The following results are as expected because the absolute errors are less than 1e-10.

const double tolerance = 1e-10;
foreach (var p in pairs)
{
    var (x, y) = p;
    var r = Math.Abs(x - y) <= tolerance;
    Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
        $"TolerateAbsError: {r}, Abs(x-y): {Math.Abs(x - y):f18}");
}

x: 0.099999999999999978, y: 0.100000000000000006,
    TolerateAbsError: True, Abs(x-y): 0.000000000000000028
x: 0.299999999999999989, y: 0.300000000000000044,
    TolerateAbsError: True, Abs(x-y): 0.000000000000000056
x: 0.999999999999999889, y: 1.000000000000000000,
    TolerateAbsError: True, Abs(x-y): 0.000000000000000111

This approach has a problem where the absolute error between two large numbers can easily exceed the minuscule number. For example, the comparison between 0.1 added to 1e6 ten times and 1e6 plus 1.0 results in false because the absolute error exceeds 1e-10.

const double tolerance = 1e-10;
double x, y;
x = y = 1e6;
for (var i = 0; i < 10; i++)
    x += 0.1;
y += 1.0;
var r = Math.Abs(x - y) <= tolerance;
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
    $"TolerateAbsError: {r}, Abs(x-y): {Math.Abs(x - y):f18}");

x: 1000000.999999999767169356, y: 1000001.000000000000000000,
    TolerateAbsError: False, Abs(x-y): 0.000000000232830644

Tolerating relative errors

You can resolve this problem by tolerating the relative error instead of the absolute error. A relative error of two numbers is the absolute error divided by the maximum of their absolute values.

The following comparison tolerant of the relative error between the two additions to 1e6 returns true because the relative error is less than 1e-10. Please notice a / b <= c forms a <= b * c to avoid division by zero.

x and y are the same as above.
var r = Math.Abs(x - y) <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
var error = Math.Abs(x - y) / Math.Max(Math.Abs(x), Math.Abs(y));
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
    $"TolerantRelativeError: {r}, RelativeError: {error:f18}");

x: 1000000.999999999767169356, y: 1000001.000000000000000000,
    TolerantRelativeError: True, RelativeError: 0.000000000000000233

This technique causes another problem. The relative error between two minuscule numbers almost equal to zero can become very large. For example, the comparison tolerant of the relative error between 1e-11 and 1e-12 returns false.

const double tolerance = 1e-10;
var x = 1e-11;
var y = 1e-12;
var r = Math.Abs(x - y) <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
var error = Math.Abs(x - y) / Math.Max(Math.Abs(x), Math.Abs(y));
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
    $"TolerantRelativeError: {r}, RelativeError: {error:f18}");

x: 0.000000000010000000, y: 0.000000000001000000,
    TolerantRelativeError: False, RelativeError: 0.900000000000000022

Torelating both errors

The equality comparison must tolerate both the absolute and relative errors of two numbers to solve those problems.

bool Equals(double x, double y, double tolerance)
{
    var diff = Math.Abs(x - y);
    return diff <= tolerance ||
           diff <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
}

The above equality comparison returns true for both of the corner cases.

const double tolerance = 1e-10;
double x, y;
bool r;
x = y = 1e6;
for (var i = 0; i < 10; i++)
    x += 0.1;
y += 1.0;
r = Equals(x, y, tolerance);
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    "+
                  $"TolerateRelativeAndAbsError: {r}");
x = 1e-11;
y = 1e-12;
r = Equals(x, y, tolerance);
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    "+
                  $"TolerateRelativeAndAbsError: {r}");

x: 1000000.999999999767169356, y: 1000001.000000000000000000,
    TolerateRelativeAndAbsError: True
x: 0.000000000010000000, y: 0.000000000001000000,
    TolerateRelativeAndAbsError: True

The myth of machine epsilon

I have arbitrarily chosen 1e-10 as the tolerance so far, but there is a mathematically rigid number misunderstood as an appropriate value in this case. It is machine epsilon.

Many programming languages define machine epsilon as the difference between 1 and the next larger floating-point number, even though the formal definition is different from it. In C#, Double.Epsilon has machine epsilon.

You can't use machine epsilon as the tolerance. When adopting Double.Epsilon as the tolerance, the comparison between the sum of ten 0.1 and 1.0 in the first example returns false. Machine epsilon is too small, and cumulative errors of arithmetic operations easily exceed it.

var one = 0d;
for (var i = 0; i < 10; i++)
    one += 0.1;
var r = Equals(one, 1.0, Double.Epsilon);
Console.WriteLine($"x: {one:f18}, y: {1.0:f18},\r\n    "+
                  $"UseMachineEpsilonAsTolerance: {r}");

x: 0.999999999999999889, y: 1.000000000000000000,
    UseMachineEpsilonAsTolerance: False

After all, the tolerance should be based on the precision necessary to your application. It also should be much larger than machine epsilon.

Conclusion

The equality comparison of floating-point numbers should take into account both the absolute and relative errors. Furthermore, the error tolerance in the comparison should be chosen based on the required accuracy in your application. If you would like to know more details of this topic, you should consult Comparing Floating Point Numbers, 2012 Edition.

Equality Comparison of Floating-Point Numbers in C# (and Others)

Related Posts