Archive for 4th April 2006

“Benchmarking inside a Virtual PC is wrong at every possible level” : Really?

Is this true? I was under the impression that virtualization technology had gotten to the point where this kind of work was fine. If not, I’ll try things out on real hardware.

Update: Trying out the n-queens problem (which I think is probably a pretty close match) with n = 14 I get these results

 

Native VMWare VPC
29.187 31.656 30.183
28.984 31.375 30.023
29.000 31.344 30.083
29.046 31.406 30.153
29.062 31.156 30.073

 

It seems to me that the virtual machines do have a measurable overhead. On the other hand, the standard deviation is less than 1/2 of 1 percent in all cases (.2% for Native and VPC, .5% for VMWare). That makes me think that virtual machines are probably okay for this kind of benchmarking (because, despite the overhead, they seem to have stable performance). Remember, one of the rules of the contest is that there’s no file io and these are console apps.

On the other other hand, the problem with my hexodoku benchmark (like most short benchmarks) is that program loading time is likely to be a very significant portion of solving order 4 Sudoku (“Hexoduko”). It would certainly make sense that virtualized overhead is greater with file access / loading and so that weighs against using a VPC.

 

Motorized inflatable pool lounger

What is with the press releases today?

Funny Team System Commercial

The short commercial at the beginning of Dr. Dobb’s TV made me laugh.

Press releases unclear on the concept

This press release for a MIME filtering antispam product is attached as a Word document file, one of the primary vectors for viruses.

 

Hexodoku Programming Contest: Win $200 Worth of The Year’s Best Books

My Hexodoku Programming Contest is live: the challenge is to generate 16 x 16 Sudoku grids. If multiple entries can consistently beat 1 second, the winner will be the program with best Big O behavior as the order of the grid scales (4^4, 5^4, 6^4, etc.). The prize is this year’s Jolt Awards Finalists in the “General Books” category:

  • “Prefactoring,” by Ken Pugh (winner)
  • “The Art of Project Management,” by Scott Berkun
  • “Ambient Findability,” by Peter Morville
  • “Producing Open Source Software,” by Karl Fogel
  • “The Best Software Writing I,” selected by Joel Spolsky
  • “Innovation Happens Elsewhere,” by Ron Goldman and Richard Gabriel

    Contest runs until May 1 2006 or, if I don’t have 4 entries by then, until I either receive 4 entries or June 1.

    Programs can be in any language that runs in an Windows XP Pro Virtual PC.