How to time code

Many Stackoverflow questions amount to nothing more than, “Which of these is faster?” or “Why is this so slow?” Very often, the person posting the question will include code that outputs some timing results. More often than not, the timing code is just wrong. Sometimes fatally so.

By the way, most “Which is faster” questions are non-starters. You shouldn’t ask before you’ve done the benchmark yourself. Read on.

If you want accurate timing information, you have to do it the right way. If you do it the wrong way, your results will be at best unreliable. Fortunately, doing things the right way is pretty easy.

But before I go any further, I feel obligated to ask the most important question: Why do you care? Before you start adding code to profile small parts of your application, you need to ask yourself if it’s really that important. Or are you just wasting your time worrying about the performance of something that is already “fast enough?” I strongly suggest that you read Eric Lippert’s, Which is faster?

Now, if you really need to time your code. . .

The information below is for .NET, and the code samples are in C#. You should follow the same techniques if you’re writing in Visual Basic .NET or Managed C++, and probably any other .NET language. The general techniques are true of pretty much any other platform as well, although the specifics will be much different.

When doing benchmarks, run them against a release build with no debugger attached. In Visual Studio, be sure to select the Release build, and either press Ctrl+F5 to run, or select Debug -> Start Without Debugging. Timings gathered from a debug build or from having the debugger attached (even in a Release build) will be inaccurate. You’ll often find that there is little difference between Debug and Release builds when the debugger is attached.

Do not use DateTime.Now, DateTime.UtcNow, or anything else based on DateTime as your time source. There are two reasons for this. First, the resolution of DateTime is somewhat questionable, but probably not much better than 15 milliseconds. Secondly, the clock can change on you. I once had a bit of code take -54 minutes (yes, it completed 54 minutes before it started) to execute because it started four minutes before 2:00 AM on “Fall back” day (Daylight Saving Time autumn adjustment date) and finished six minutes later. So start time was 01:56:00 and end time was 01:02:00. You don’t control the system clock, so don’t depend on it.

Use the System.Diagnostics.Stopwatch class for all elapsed time measurements. That’s based on the Windows high performance timer, which is the most accurate interval timer you’re likely to get on a Windows box unless you have special hardware. Accept no substitutes.

With Stopwatch, timings are easy. First, add this using statement to the top of your source file:

    using System.Diagnostics;

Then, surround the code you want to time with a Stopwatch:

    Stopwatch sw = Stopwatch.StartNew(); // creates and starts the stopwatch
    //
    // put the code you want to time here
    //
    sw.Stop();  // stop the watch.
    Console.WriteLine("Elapsed time = {0} milliseconds", sw.ElapsedMilliseconds);

That’s all there is to it!

I usually like to report times in milliseconds (1/1000 second), simply because the usable range is reasonable. If I’m timing something that takes less than a millisecond to run, I need to do it many times to get a good average. A million iterations of any non-trivial loop will likely take longer than a few milliseconds. On the same note, a full day is 86,400 seconds or 86,400,000 milliseconds: a number that is still reasonably easy to work with. If I’m timing something that takes longer than a full day to run, I might decide to use seconds as my comparison.

One other nice thing about Stopwatch is that you can have many of them, each timing a discrete part of your program. For example:

    Stopwatch totalTime = Stopwatch.StartNew();

    Stopwatch pass1Time = Stopwatch.StartNew();
    DoPass1();
    pass1Time.Stop();

    Stopwatch pass2Time = Stopwatch.StartNew();
    DoPass2();
    pass2Time.Stop();

    totalTime.Stop();

You then have separate values for the total time and the time to execute each pass.

If you’ve done all of that and you still haven’t found an answer, then post a question. But include your timing code so that those of us who are willing to help you find an answer aren’t groping around in the dark. And be sure to mention that you ran in Release mode without the debugger attached.

1 comment to How to time code

  • mikeb

    A few things that readers might want to keep in mind:

    If you’re timing .NET code, you should run an untimed pass of the function that contains the code you’re interested in benchmarking. Otherwise you may be including the time it takes to JIT some or all of the code under test, which probably isn’t what you want. This probably applies to Java as well.

    Another thing that can cause misleading results with simple benchmarks is that often the code being timed is simplified to the point where the compiler or jitter can optimize away many of the operations that you’re intending to time. I’ve seen confusion in several Stackoverflow questions/answers because of this.

    Oh, and if you’re benchmarking C/C++ code, please do it with an optimized build. There’s usually not much point benchmarking non-optimized code.

Categories

A sample text widget

Etiam pulvinar consectetur dolor sed malesuada. Ut convallis euismod dolor nec pretium. Nunc ut tristique massa.

Nam sodales mi vitae dolor ullamcorper et vulputate enim accumsan. Morbi orci magna, tincidunt vitae molestie nec, molestie at mi. Nulla nulla lorem, suscipit in posuere in, interdum non magna.