Sort of example with only partial processing (because you didn't show any). You can also pass back all the data in the child's return hash, I didn't just because I didn't know how you wanted to process it.

See Re^3: Pre-Forking Daemon with Parallel::ForkManager for the Forking::Amazing module. 

#!/usr/bin/perl

use strict; # https://www.perlmonks.org/?node_id=11154241
use warnings;
use Forking::Amazing;
use Time::HiRes qw( time );
use LWP::UserAgent;

my $ua = LWP::UserAgent->new;
my %answers;

Forking::Amazing::run
  23, # maxforks, less than two dozen
  sub # runs in child
    {
    my $starttime = time;
    my ($url) = @_;
    print "child debug starting $url\n";
    $ua->agent($url);
    my $req = HTTP::Request->new(GET => $url);
    my $res = $ua->request($req);
    my $status = $res->status_line($req);
    my $data = $res->content;
    print "child debug ending $url\n";
    return { len => length $data, status => $status,
      status => $status, took => sprintf "%.3f seconds", time - $start
+time };
    },
  sub # runs in parent with results from child
    {
    my ($url, $hashref) = @_; # url child, hashref from child
    $answers{$url} = $hashref;
    },
  map 'https://' . tr/\n//dr, <DATA>; # list of URLs to process
use Data::Dump 'dd'; dd \%answers;

__DATA__
gap.com
amazon.com
ebay.com
wunderground.com
imdb.com
google.com
nosuchurl.com
underarmour.com
disney.com
espn.com

[download]

Outputs: 

child debug starting https://gap.com
child debug starting https://amazon.com
child debug starting https://ebay.com
child debug starting https://wunderground.com
child debug starting https://imdb.com
child debug starting https://google.com
child debug starting https://nosuchurl.com
child debug starting https://underarmour.com
child debug starting https://disney.com
child debug starting https://espn.com
child debug ending https://nosuchurl.com
child debug ending https://wunderground.com
child debug ending https://google.com
child debug ending https://disney.com
child debug ending https://gap.com
child debug ending https://espn.com
child debug ending https://ebay.com
child debug ending https://underarmour.com
child debug ending https://amazon.com
child debug ending https://imdb.com
{
  "https://amazon.com"       => { len => 724660, status => "200 OK", t
+ook => "1.575 seconds" },
  "https://disney.com"       => { len => 548346, status => "200 OK", t
+ook => "0.547 seconds" },
  "https://ebay.com"         => { len => 468521, status => "200 OK", t
+ook => "0.832 seconds" },
  "https://espn.com"         => { len => 1474156, status => "200 OK", 
+took => "0.649 seconds" },
  "https://gap.com"          => { len => 495420, status => "200 OK", t
+ook => "0.624 seconds" },
  "https://google.com"       => { len => 19623, status => "200 OK", to
+ok => "0.352 seconds" },
  "https://imdb.com"         => { len => 947107, status => "200 OK", t
+ook => "2.467 seconds" },
  "https://nosuchurl.com"    => {
                                  len => 168,
                                  status => "500 Can't connect to nosu
+churl.com:443 (Name or service not known)",
                                  took => "0.048 seconds",
                                },
  "https://underarmour.com"  => { len => 491776, status => "200 OK", t
+ook => "1.015 seconds" },
  "https://wunderground.com" => { len => 168647, status => "200 OK", t
+ook => "0.257 seconds" },
}

[download]

Of course, after it's working the debug lines can be removed :)

In reply to Re: IPC::Open, Parallel::ForkManager, or Sockets::IO for parallelizing? by tybalt89
in thread IPC::Open, Parallel::ForkManager, or Sockets::IO for parallelizing? by mldvx4

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.