Building a multi-threaded program is much harder than building a single threaded one. You can't just spread on a little multi-threading to make it go faster.

Actually, often you can.

As a simplistic example, take the problem in Averaging Elements in Array of Array, and Ikegami's solution as a starting point. If you wrap the first (nested) loops of that solution up into a subroutine that returns the array of sums, then divides them to produce the averages, you have a single threaded solution that will run in time T.

Now assume you have a machine that has 4 cpus/cores. If you call that same subroutine, passing a subpartiton of the total dataset, 4 times--each time in a different thread:

#! perl -slw
use strict;
use threads;
use threads::shared;

sub sumEm{
    my $tid = threads->tid;
    my( $AoARef, $first, $n ) = @_;
    print "$tid: $first - ", $first + $n - 1;
    my @sums = ( 0 ) x 4;
    for my $i ( $first .. $first+$n-1 ) {
        $sums[ $_ ] += $AoARef->[ $i ][ $_ ] for 0 .. 3;
    }
    return @sums;
}

our $N ||= 1e6;
our $T ||= 4;

my @AoA:shared;
$#AoA = $N;

for( 0 .. $N -1 ) {
    $AoA[ $_ ] = &share([]);
    @{ $AoA[ $_ ] } = map rand( 100 ), 1 .. $T;
}

my $perThread = int $N / $T;
my @threads = map{
   threads->create( \&sumEm, \@AoA, $_* $perThread, $perThread )
} 0 .. $T-1;

my @aves = ( 0 ) x 4;
for my $thread ( @threads ) {
    my @subtotals = $thread->join;
    $aves[ $_ ] += $subtotals[ $_ ] for 0 .. 3;
}

$_ /= $N for @aves;

print "@aves";
[download]

The chances are that it will arrive at the same answer in something less than T/3 the time. (I don't have a quad cpu machine on which to prove this!).

I've seen many projects collapse from underestimating the work involved.

I'm gonna make three (totally hairy-arsed, unfounded, speculative) guesses about those projects:

• They weren't in Perl using iThreads.

  iThreads are imperfect, badly documented, and widely misunderstood.

  But, they have some very unique properties that you will see being copied more and more as multi-core machines and threading become ubiquitous.

  Indeed, I think, Artur Bergman will, belatedly, be recognised as having contributed something quite special to the world of computing. Something so special, that--when the world catches up with him--will eventually put him up there with the likes of Lovelace, Turing, Knuth and Berners-Lee.

• They were in some language in which OO-techniques predominated, or were unavoidable.

  It's not that OO and threads don't mix--well, they don't in Perl, but that for other reasons. It is that people will insist on trying to share objects across threads. This creates problems--which relate to the 'horizontal partitioning' and/or 'synchronisation' problems below, but compound them.

  There is no problem, barring an (artificial) Perlish limitation, with passing objects between threads. The problem arises when trying to code classes such that their instances and methods can be called concurrently on multiple threads. If you avoid that scenario, about 80% (speculative) of the problems simply disappear.

• The architects were writing their first multithreaded project, and made the beginner's mistake of trying to:
  • Partition the problem horizontally rather than vertically.

    That is, they tried to partition the algorithm(s) across threads, rather than the data.

  • And/or: tried to synchronise the threads.

    This is the threading FAQ error number one. They attempted to share data, rather than communicate information, across threads.

Threads needn't be hard. Certainly no harder, and often easier than forking. There are a raft of commonly cited, difficult to solve, problems that bug threading. And there has been much research, many papers, and a raft-squared proposed, usually complex, or (computationally and/or algorithmically) costly solutions to those problems.

But the simplest solution is invariably overlooked: avoid them!

That is, rather than trying to come up with complex and costly solutions to the well-known problems with locking, sharing and synchronisation--just don't use algorithms and techniques that require them.

iThreads are a damn good start down that road.