maybench underway

The maybench project (formerly checkquick) is now underway. Maybench is a tool for comparing the performance between two versions of the same program, on a series of benchmarks that you design. Maybench aims to be easy to use, almost as easy as running time your-program arg1..arg2. Ideally, it should be very straightforward for outsiders to write timing tests for your programming project and contribute them as part of your performance testing suite.

We have a Google Code page, a mailing list and a darcs repository:

  darcs get http://code.haskell.org/maybench

The repository basically consists of the preliminary code written by the tehgeekmeister and also some new code by ertai (see darcs-benchmark). I'm also hoping that some of the code written for nobench can be used, for example, to generate fancy reports. Right now, running maybench looks like this:

% dist/build/maybench/maybench 'sleep 5' 'sleep 3'
"sleep 3" took 60.0% of the time "sleep 5" took.

As a first priority, we're going to get maybench useable for benchmarking darcs. After that we'll start thinking of how to generalise it, so that it can be used for the Haskell benchmark suite, for example, or for your software.

Interested? Come join us!