Precision of Floating-Point Data Types

C# Tutorials - Herong's Tutorial Examples

∟Precision of Floating-Point Data Types

This section provides a tutorial example on how to compare precisions of floating-point data types: 'float', 'double', and 'decimal'

Before looking at the output of presented in this section, please take a guess at final values of these expressions:

((float)1/3)*3
((double)1/3)*3
((decimal)1/3)*3

If you are an entry level programmer, you may guess that:

((float)1/3)*3 = 1
((double)1/3)*3 = 1
((decimal)1/3)*3 = 1

If you are an experienced programmer, you may guess that:

((float)1/3)*3 = 0.9999999
((double)1/3)*3 = 0.999999999999999
((decimal)1/3)*3 = 0.9999999999999999999999999999

But both answers are wrong. In C#, you will get:

((float)1/3)*3 = 1
((double)1/3)*3 = 1
((decimal)1/3)*3 = 0.9999999999999999999999999999

The following tutorial example demonstrates this answer in a slightly different way:

// Precision.cs
// Copyright (c) 2006 HerongYang.com. All Rights Reserved.

using  System;
class Precision {
   public static void Main() {
      float f;
      double d;
      decimal m;
      for (int i=1; i<=2; i++) {
         f = (float)i/3;
         d = (double)i/3;
         m = (decimal)i/3;
         Console.WriteLine("Testing {0}/3:", i);
         Console.WriteLine(" f = {0}", f);
         Console.WriteLine(" d = {0}", d);
         Console.WriteLine(" m = {0}", m);
         Console.WriteLine(" f*3 = {0}", f*3);
         Console.WriteLine(" d*3 = {0}", d*3);
         Console.WriteLine(" m*3 = {0}", m*3);
         Console.WriteLine(" (double)f*3 = {0}", (double)f*3);
         Console.WriteLine(" (decimal)f*3 = {0}", (decimal)f*3);
         Console.WriteLine(" (decimal)d*3 = {0}", (decimal)d*3);
         Console.WriteLine(" (double)((float)i/3)*3 = {0}",
            (double)((float)i/3)*3);
      }
   }
}