I can't do anything right...

(tl;dr? Summary at end)

So, I've been working on a rewrite of the most time-critical component of our code. I've been begging for a chance to do this pretty much the entire time I've been working here, and I've finally been given the chance.

There are several versions of this code that already exist -- so why write it again? Well, our options are:

• An incomprehensible mess of Fortran that runs (presumably) very quickly, but is one of the greatest examples of write-only code I've ever seen.
• A huge, featureful and highly-modular C++ implementation that takes about 3-4x longer to run.
• A slimmer C/C++ implementation (C++ design with a lot of C-isms) that was developed by a collaborator. It strikes a compromise between the previous two in both size and speed.

Back in December I modified a version of the third implementation to run in the same amount of time as the original Fortran implementation. It was an ugly hack, however, and we'll speak of it no more.

As an aside, I am completely sick of "Fortran is faster than C/C++" arguments. How you write the code affects speed, but the distinction between these languages (for line-by-line equivalent code) does not. Most modern compilers for all of these languages share a common backend, so STFU and GTFO. C++ provides a lot of tools to help you write better and cleaner code, but it's the responsibility of the programmer to understand the cost associated with each of these optional tools.

Anyway... I've been working on a new version of the code. The underlying philosophy is to never sacrifice any performance, then make the cleanest and most extensible code possible.

I completely missed my target. My goal has been to match the performance of the original Fortran code, which takes 20 seconds to run on a standard example problem. Not only did I miss my target, I missed it by a factor of three. Ouch.

My new code runs in 6.5 seconds.

Oh. :)

I did the following:

• Obsessively tighten and optimize the memory layout.
• Restructure the iteration order for better cache performance.
• Write the code in a way such that the compiler can do a decent job rewriting the math with SIMD (SSE2) instructions.
• Exploit a near-symmetry in the kernel (my pride and joy, the only thing that could still be done to actually reduce the number of flops).

Oh, and to top it off, my code is very readable and only clocks in at just over 2k lines, including plentiful comments. Furthermore, nearly half of that is specific to the problem (the acoustic dirichlet kernel, rather than the method). I think the equivalent functionality from the large C++ code is at around 20k lines, and nigh-incomprehensible thanks to a few overly-aggressive optimization sweeps (in which, admittedly, I took part).

I know that, often, the "let's re-write it from scratch" argument is a bad one, but I think I've made my case this time around.

Summary: Rewriting critical code, and I did better than I expected. Not only did I make my performance target, but I managed to beat it by a factor of three. Yay.