Must We Multi-thread? (Part 1)

Most of my work for the past years have been on multi-threaded applications. That is, most of the applications that I've worked on have been multi-threaded, and sometimes clustered. And I'm no stranger to small-scale concurrent programming (imagine a fourth cousin three times removed!) And I've come to dislike this multi-threading. Not because it's hard (I like this sort of hard,) but rather because it tends to make your code messy, hard to understand, hard to debug, in one word ugly To top it all off, the performance gain is not always even close to what you'd expect. On the other hand, the processor clock-rates are not on the mad rise anymore. All that remains for processor makers in this race is to increase the number of threads/cores/chips. First it was two, four or more CPUs in a system, but the desktop market didn't buy it. Then came SMT (Symmetric Multi-threading?) a.k.a Hyper-Threading, and now dual- and quad-core CPUs. I seem to remember reading somewhere that around 70% of the CPUs sold by Intel in 2006 were multi-core. I'm sure AMD has similar stats on its multi-core sales. What that means for me the programmer is that I can no longer think of multi-threading in terms of "one for IO, one for actual work." When you have a single CPU available, that means the only thing you can parallelize is IO (disk, network, graphics, sound.) (OK, I know that you can do a lot of "actual work" on the GPU too, but let's not complicate things anymore that they need to be.) I have to really think about partitioning my work over 2, 4, 6 (don't!) or even more hardware threads. Fortunately (or unfortunately) ordinary, end-user programs have yet to need to be able to scale up into the hundreds or thousands of CPUs (at least, not at a fine-grained level of parallelism) so we can still retain the same algorithms (mostly) and only restructure the programs. More on this later.