Howdy monks! So I have been trying to code a script to fetch info for game, and I found with out threads this is simply to slow to be effective. However, In an attempt to make things faster I added threading, but now when I run the script it quickly shoots up to 4gb memory then crashes. Now I am completely new to this so some one school me on what I'm doing wrong so this never happens again. note, the var $names is 16k lines long 

use threads;
use warnings;
use LWP::Simple;


$names='';
my $re69='((?:[a-z0-9_]+))';
open (lookup, '>>rstlookup.txt');

my @a = ();
my @b = ();


print "Starting main program\n";

my $nb_process = 16832;
my $nb_compute = 20;
my $i=0;
my @running = ();
my @Threads;
while (scalar @Threads < $nb_compute) {
     @running = threads->list(threads::running);
    while ($names =~ m/$re69/isg)
    {
        if (scalar @running < $nb_process) {
            $name = $1;
             my $thread = threads->new(\&lookup);
            push (@Threads, $thread);
            my $tid = $thread->tid;
        }
    @running = threads->list(threads::running);
    foreach my $thr (@Threads) {
        if ($thr->is_running()) {
            my $tid = $thr->tid;
        }
        elsif ($thr->is_joinable()) {
            my $tid = $thr->tid;
            $thr->join;
        }
    }
}
    @running = threads->list(threads::running);
    $i++;
}
while (scalar @running != 0) {
    foreach my $thr (@Threads) {
        $thr->join if ($thr->is_joinable());
    }
    @running = threads->list(threads::running);
}


sub lookup {
    my $name2 = $name;
    my $lookup = get("http://rscript.org/lookup.php?type=track&time=62
+899200&user=".$name2."&skill=all");
    print "Looking up $name2...\n";
    $reg1='(ERROR)';
    if ($lookup) {
        my $re1='(gain)';
        my $re2='(:)';
        my $re3='(Overall)';
        my $re4='(:)';
        my $re5='(\\d+)';
        my $re6='(:)';
        my $re7='(\\d+)';
        my $re=$re1.$re2.$re3.$re4.$re5.$re6.$re7;
        if ($lookup =~ m/$re/isg) {
            print lookup "$name2 $7\n";
        }
    }
    if ($lookup =~ m/$reg1/isg) { print lookup "$name2 doesn't exist \
+n" }
    else{ print lookup "$name2 0\n"; }
}
close (lookup);

[download]

In reply to Consumes memory then crashs by allhellno

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.