in reply to Bandwidth Usage?

You might want to use a shared hash to do the winnowing, like so (untested):
use strict; use warnings; use threads; use threads::shared; use Thread::Queue; my $queue = Thread::Queue->new; my %urls = (); share(%urls); my $url = 'http://www.perlmonks.org/'; my $visited = do { lock(%urls); $urls{$url}++ }; if (!$visited) { $queue->enqueue($url); }

Update: added locking

Replies are listed 'Best First'.
Re^2: Thread::Queue Internals/Duplicates
by ramblinpeck (Sexton) on Feb 03, 2006 at 21:01 UTC
    Doh, you must have caught me updating :). I figured out the Threads:Queue thing, with pretty much the suggestion you gave. Thanks,