Sector 0

Mono vs. Microsoft : bout 2

After posting the previous blog entry on Mono IL code, Marek Safar and Miguel de Icaza from the Mono team noticed that I had made the rather embarassing mistake of compiling the code in debug-mode when compiling with Microsofts compiler. That left me with quite a red face, but now is the time to do it properly.

For this test I have written a small application that computes prime numbers using a rather inefficient but simple algorithm. The program calculates the prime numbers between 2 and 100,000 (yeah, I just love primes). This is done 1,000 times and the average execution time is displayed.

The application consists of this single class:

class Program
{
	static void Main(string[] args)
	{
		int n = 1000;
		double sumMs = 0;
		for (int i = 0; i < n; i++)
		{
			DateTime dtStart = DateTime.Now;
			Run();
			TimeSpan tsTotal = DateTime.Now - dtStart;
			sumMs += tsTotal.TotalMilliseconds;
			Console.Write(string.Format("Number {0}\\r", i));
		}
		Console.WriteLine(string.Format("Total execution time = {0}", sumMs));
		Console.WriteLine(string.Format("Average execution time = {0}", sumMs / (double)n));
	}
 
	private static void Run()
	{
		for (long l = 2; l < 100000; l++)
			IsPrime(l);
	}
 
	private static bool IsPrime(long primeCandidate)
	{
		//two is by definition a prime number, 1 is not however
		if (primeCandidate == 2)
			return true;
		//throw an exception if the number is less than 2
		if (primeCandidate < 2)
			throw new ArgumentException("Prime candidates cannot be less than 2");
		//throw away even numbers
		if ((primeCandidate % 2) == 0)
			return false;
 
		long max = (long)(Math.Sqrt(primeCandidate) + 1);
		for (long divisor = 3; divisor < max; divisor += 2)
			if ((primeCandidate % divisor) == 0)
				return false;
		return true; //it is a prime
	}
}

The program is compiled with both csc.exe (Microsoft) and gmcs.exe (Mono) with the following commands:

csc /optimize+ /target:exe /out:perftest.ms.exe Program.cs
gmcs Program.cs -optimize+ -target:exe -out:perftest.mono.exe

Surprisingly this yields two executables that are exactly the same size, but that might be a coincidence; On average the Mono-generated binaries is half to two-thirds the size of the same code compiled with Microsofts compiler, so Mono is still the best choice when it comes to mobile devices.

The executables where run 5 times each, on each runtime. The results are shown below in milliseconds of execution time.

At closer inspection it seems that the Mono-code is sligtly faster on Microsofts runtime, and the Microsoft-code is slightly faster on the Mono runtime. This may be another coincidence, but nonetheless the discrepancies are too small to be anything other than statistical anomalies and nothing conclusive can be said from the data.

The only conclusion we can draw from the above data is, that apparently the Mono runtime is somewhat slower than Microsofts ditto. I write “apparently” since the data set and the number of algorithms tested are not enough to provide us with enough statistical data to make that assumption. For example I have observed that TCP-connections perform just as well on Monos runtime as Microsofts, and the differences in execution time in the above sample are so small that they can be disregarded as measurement errors.

But the guys working on Mono is well aware that their runtime does not perform as well as Microsofts. Currently they are all busy implementing stuff from .NET 3.0 and 3.5, but I think that we will see some improvements on performance in the near future.

And let’’s not forget how important the hard work these guys are doing is. Having a multi-platform .NET environment is so important for the future of software development (.NET languages are some of the most popular languages) that I am more than willing to overlook a few differences in performance and functionality. Besides, performance isn”t everything. In most software systems - even enterprise level - the processors are mostly idle.