Here is what I would do , partition the url list upfront into different storable files, so when you fork you're only sharing a single filename, then later the parent process unifies the results of the child process ... only parent partitions jobs because only parent spawns children

#!/usr/bin/perl --
## 
## 
## 
## perltidy -olq  -csc -csci=3 -cscl="sub : BEGIN END " -otr -opr -ce 
+-nibc -i=4 -pt=0   "-nsak=*"
## perltidy -olq  -csc -csci=10 -cscl="sub : BEGIN END if " -otr -opr 
+-ce -nibc -i=4 -pt=0   "-nsak=*"
## perltidy -olq  -csc -csci=10 -cscl="sub : BEGIN END if while " -otr
+ -opr -ce -nibc -i=4 -pt=0   "-nsak=*"
#!/usr/bin/perl --
use strict;
use warnings;
use Data::Dump qw/ dd /;

Main( @ARGV );
exit( 0 );

sub Main {
    my @files = StorePartitionUrls( GetInitialUniqueUrls() );
    ForkThisStuff( @files );
    UnifyChildResults( 'Ohmy-unique-hostname-urls-storable', @files );
} ## end sub Main

sub GetInitialUniqueUrls {
    my @urls;
    ...
    return \@urls;
} ## end sub GetInitialUniqueUrls

sub ForkThisStuff { ## spawn kids with one file, wait, whatever
    for my $file ( @files ) {
        EachChildGetsItsOwn( $file );
    }
} ## end sub ForkThisStuff

sub ForkThisStuff {
    for my $file( @files ){
## something forking here
        EachChildGetsItsOwn( $file );
    }
}

sub StorePartitionUrls {
    my( $urls , $partition , $fnamet, ) = @_;
    $partition ||= 100;
    $fnamet    ||= 'Ohmy-candidate-urls-%d-%d-storable';
    my @files;
    while( @$urls ){
        my @hundred = splice @$urls, 0, $partition ;
#~         my $file    = "Ohmy-".int( @$urls ).'-'.int( @hundred ).'-s
+torable';
        my $file    = sprintf $fnamet, int( @$urls ), int( @hundred );
        lock_store \@hundred, $file;
        push @files, $file;
    }
    return @files;
} ## end sub StorePartitionUrls

__END__
[download]

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.