rodeo:

When I'm trying to make code faster, threading isn't the first tool I reach for. Usually you'll have the best results if you improve your algorithm(s). You don't show your code or algorithm, so I can't offer any suggestions there. However, you mention that you have five interlaced for loops. I think I'd start looking at those loops to see if you can tune things up there.

Verify that you're using data structures suited to your task. If you constantly find yourself looping through an array to find a data element, perhaps it would be better to use a hash so you can access data by name. Frequently that can remove some loops from your code, and can be a dramatic speed increase.

Once I'm happy with my algorithm and data structures, the next thing I try when I need to make something faster is to tune up the algorithm I'm currently using: First, I move calculations out of the most frequently executed loop(s) when possible. In the example below (first section), rather than compute sin($j) 100*100*100 times, we compute it only 100 times. Similarly we compute cos($k) only 100*100 times. The overall savings can add up quite a bit!

Next, if you have some expensive subroutines that you call frequently with the same parameters, you might find Memoize handy: It will store the results of a call and reuse the result when you call it again with the same value(s). As the second example shows, it can give a big win for only two lines of code.

There are many other optimization tricks you can use, too.

Now I must confess: I've never used perls code profiler, though I've been meaning to get around to it. But find and use a good profiler to find out what your code is doing that takes the most time. That will show you where to focus your attention. If you use a profiler rather than guessing, it can point you to some surprising places. (Most of my speed-critical code is in C/C++, where I *do* profile my code.) You might, for example, find that half your execution time is spent formatting your data for printing your progress to a log. In that case, you might log less information and save a lot of time. (As an example, see how loop3 is *so* much slower than loop1?)

Usually I don't reach for threading/forking until I've used the above techniques. If the problem is easy to split into independent pieces, then threading can be easy enough and give you some good results. But threading can often be difficult to apply to some problems. If you split your problem incorrectly, then you'll find that your program takes *longer* to run. If you need assistance with threading, you'll want to appeal to a monk who uses threading more than me (e.g., BrowserUk).

$ perl 1051579.pl
      s/iter loop3 loop1 loop2
loop3   17.5    --  -88%  -92%
loop1   2.08  740%    --  -36%
loop2   1.33 1210%   56%    --


no   memoize: 30.9833 wallclock secs (30.98 usr +  0.00 sys = 30.98 CP
+U) @ 1613.94/s (n=50000)
with memoize: 0.816253 wallclock secs ( 0.82 usr +  0.00 sys =  0.82 C
+PU) @ 60975.61/s (n=50000)

$ cat 1051579.pl 
#!/usr/bin/perl
use strict; use warnings;
use Benchmark qw(:all :hireswallclock);
#use Memoize;

my ($cnt1, $cnt2);

sub loop1 {
    $cnt1=0;
    for my $j (0 .. 100) {
        for my $k (0 .. 100) {
            for my $l (0 .. 100) {
                $cnt1 += sin($j) + cos($k) + sqrt($l);
            }
        }
    }
}

sub loop2 {
    $cnt2=0;
    for my $j (0 .. 100) {
        my $sin_j = sin($j);
        for my $k (0 .. 100) {
            my $cos_k = cos($k);
            for my $l (0 .. 100) {
                $cnt2 += $sin_j + $cos_k + sqrt($l);
            }
        }
    }
}

open my $FH, '>', '/dev/null';

sub loop3 {
    $cnt1=0;
    for my $j (0 .. 100) {
        for my $k (0 .. 100) {
            for my $l (0 .. 100) {
                $cnt1 += sin($j) + cos($k) + sqrt($l);
                printf $FH "% 3u % 3u % 3u : %f\n", $j, $k, $l, $cnt1;
            }
        }
    }
}
cmpthese 5, {
    loop1 => q[ loop1( ) ],
    loop2 => q[ loop2( ) ],
    loop3 => q[ loop3( ) ],
};

# double-check that the routines do the same thing!
print "**** $cnt1 != $cnt2 ****\n" if abs($cnt1-$cnt2)>.0001;
print "\n\n";

no warnings 'recursion';

# hideously expensive subroutine
sub glarf {
    my $t = shift;
    return 1 if $t <2;
    return 2 if $t <3;
    return glarf($t-1)+glarf($t-2)+glarf($t-3);
}
my $iter=50000;
my $fibsub = sub {
    glarf(10); #glarf(20); #glarf(30); glarf(40); glarf(50); glarf(100
+);
};

timethis($iter, $fibsub, 'no   memoize' );

# These two lines of code make it *much* faster
use Memoize;
memoize('glarf');

timethis($iter, $fibsub, 'with memoize' );
[download]

Update: It seems rodeo has posted the code while I was writing this. But I'm stopping for now because I've got to get to work...

...roboticus

When your only tool is a hammer, all problems look like your thumb.

Or somebody have an other idea how I can release this problem?

Post the loop code.

How big are:

1. @ySurf
2. @alfas
3. $data->{count}

How big, or what range of sizes does $center->{count} take?

The sizes of @ysurf and @alfas are round about 10...15. $data->{count} more than 50.000.000. The count of center is thousands and more

Your code is full of silly inefficiencies; like these 3 calculations:

$data->{magX}[ $dataIdx ] * cos( $data->{phX}[ $dataIdx ] )
$data->{magY}[ $dataIdx ] * cos( $data->{phY}[ $dataIdx ] )
$data->{magZ}[ $dataIdx ] * cos( $data->{phZ}[ $dataIdx ] )
[download]

being performed for all 50 million $data points; and all 50 million being repeated for every one of the "thousands" of $center points. That's 3 * "thousands" * "50.000.000" of redundant, repeated calculations.

Also, whilst I won't pretend to fully understand your code in isolation of what TransfNorVec(), coordSystTransf() & Check() subs do, it seems fairly clear you are checking a bunch of 3D points against another bunch of 3D points and accumulating statistics about how many of the former are within some distance of some of the latter.

And you are going about it by brute force.

With a clearer picture of the data (in $data (what an awful name) and $center), it ought to be possible to order (sort or structure -- perhaps 3-dimensionally) them, so that for any given $input only a small subset of the 'thousands by millions' of comparisons (and associated calculations) need be done.

For example, if the point's X value is 50 and the radius of the compare is 5, there is no point in comparing against any other point who's X-value is outside of 45 thru 55. Sub-setting the larger range early could save billions of calculations, comparisons, object constructions and function calls.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Where does $data come from?

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Ignore this.