Herong's Tutorial Notes On C# - Part B
Dr. Herong Yang, Version 2.02

Precision of 'float', 'double', and 'decimal'

Before looking at the output of the following program, please take a guess at the values of these expressions:

  • ((float)1/3)*3
  • ((double)1/3)*3
  • ((decimal)1/3)*3

If you are an entry level programmer, your guess would be:

  • ((float)1/3)*3 = 1
  • ((double)1/3)*3 = 1
  • ((decimal)1/3)*3 = 1

If you are an experienced programmer, your guess would be:

  • ((float)1/3)*3 = 0.9999999
  • ((double)1/3)*3 = 0.999999999999999
  • ((decimal)1/3)*3 = 0.9999999999999999999999999999

But both answers are wrong. In C#, you will get:

  • ((float)1/3)*3 = 1
  • ((double)1/3)*3 = 1
  • ((decimal)1/3)*3 = 0.9999999999999999999999999999

The following program demonstrates this answer in a slightly different way:

using  System;
class Precision {
   public static void Main() {
      float f;
      double d;
      decimal m;
      for (int i=1; i<=2; i++) {
         f = (float)i/3;
         d = (double)i/3;
         m = (decimal)i/3;
         Console.WriteLine("Testing {0}/3:", i);
         Console.WriteLine(" f = {0}", f);
         Console.WriteLine(" d = {0}", d);
         Console.WriteLine(" m = {0}", m);
         Console.WriteLine(" f*3 = {0}", f*3);
         Console.WriteLine(" d*3 = {0}", d*3);
         Console.WriteLine(" m*3 = {0}", m*3);
         Console.WriteLine(" (double)f*3 = {0}", (double)f*3);
         Console.WriteLine(" (decimal)f*3 = {0}", (decimal)f*3);
         Console.WriteLine(" (decimal)d*3 = {0}", (decimal)d*3);
         Console.WriteLine(" (double)((float)i/3)*3 = {0}", 
            (double)((float)i/3)*3);         
      }
   }
}

Output:

Testing 1/3:
 f = 0.3333333
 d = 0.333333333333333
 m = 0.3333333333333333333333333333
 f*3 = 1
 d*3 = 1
 m*3 = 0.9999999999999999999999999999
 (double)f*3 = 1.00000002980232
 (decimal)f*3 = 0.9999999
 (decimal)d*3 = 0.999999999999999
 (double)((float)i/3)*3 = 1
Testing 2/3:
 f = 0.6666667
 d = 0.666666666666667
 m = 0.6666666666666666666666666667
 f*3 = 2
 d*3 = 2
 m*3 = 2.0000000000000000000000000001
 (double)f*3 = 2.00000005960464
 (decimal)f*3 = 2.0000001
 (decimal)d*3 = 2.000000000000001
 (double)((float)i/3)*3 = 2

Question 1: Why ((float)1/3)*3 = 1?

Don't try to answer it with the possibility that the compiler is smart, and optimizes (/3)*3 as *1. I have tried to get the /3 part from a variable like the demonstration program, from a method, or even from another class that compiled separately. They all give me the same result.

I believe that the answer is related to the storage algorithm of the float data type. But I don't have time to figure it out yet.

Question 2: If ((float)1/3)*3 = 1, then why ((decimal)1/3)*3 = 0.9999999999999999999999999999?

I also don't have a clear answer yet. But I believe that the answer is also related to the storage algorithm of the data types.

No matter what the answer is, this behavior is telling us that "decimal" is not a natural extension of "float", like "double" extending "float".

If you are a designer, don't ask your developers to change "double" to "decimal" immediately. Do some experiments first.

Question 3: If ((float)1/3)*3 = 1, and ((decimal)1/3)*3 < 1, then is the float type more accurate that the decimal type?

I will leave this question to you to answer.

Question 4: If f is (float)1/3 = 0.3333333, then why (double)f*3 = 1.00000002980232?

I am expecting (double)f*3 = 0.9999999, because casting f into "double" will force the expression to be evaluated in double precision, giving 0.9999999. What do you think?

Question 5: If (double)f*3 = 1.00000002980232, why (double)((float)1/3)*3 = 1?

The difference here is that (float)1/3 is evaluated first and stored in a variable f.

My guess to what happened here is that all floating data expressions are evaluated in double precision. What do you think?

Dr. Herong Yang, updated in 2002
Herong's Tutorial Notes On C# - Part B - Precision of 'float', 'double', and 'decimal'