nwboy74 has asked for the wisdom of the Perl Monks concerning the following question:

Why isn't this working? When I run it, it hangs when it gets to "print WRITER $content". This is a simplified version of a bigger program I'm troubleshooting. Disregard the reason I'm forking or whatnot.

use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use HTTP::Message;

my $ua = LWP::UserAgent->new();
my $iteration = 1;
open(URLS, "SomeFileOfURLs");
while (<URLS>) {
	my $url = $_;
	chomp $url;
	&get_url($url, $iteration);
	print "($iteration) $url\n";
	sleep 15;
	$iteration++;
}
close URLS;

sub get_url {
	my ($url, $iteration) = @_;
	pipe(READER, WRITER);
	if (my $pid = fork) {
		close WRITER;
		waitpid($pid, 0);
		my @data = (<READER>);
		open(FH, ">SomeDirectory\\$iteration");
		print FH @data;
		close READER;
		close FH;
	} elsif (defined $pid) {
		close READER;
		my $request = HTTP::Request->new("GET", $url);
		my $response = $ua->request($request);
		my $content = $response->content();
		print WRITER $content;
		undef $request;
		undef $response;
		close WRITER;
		exit;
	}
}

The program is supposed to take a url, download the page from the Internet, then return the results.

Replies are listed 'Best First'.
Re: Forking Issue
by ikegami (Patriarch) on Oct 20, 2010 at 21:30 UTC

    The child keeps writing and writing to the pipe, but nothing is emptying the pipe.

    Once the pipe becomes full, the child blocks until the pipe becomes empty enough to handle the write.

    Of course, that's never gonna happen in your program since the "reader" is spending its time waiting for the child to end rather than emptying the pipe.

    You have what is called a deadlock. The child is blocked waiting for the parent to empty the pipe, the parent is blocked waiting for the child to exit.

    Change 

    waitpid($pid, 0);
my @data = (<READER>);

    [download]

    to 

    my @data = (<READER>);
waitpid($pid, 0);

    [download]
[reply]
[d/l]
[select]

      Ah. Thanks! I didn't know the pipe buffer was so small. Totally makes sense now.

[reply]
Re: Forking Issue
by SuicideJunkie (Vicar) on Oct 20, 2010 at 19:20 UTC

    You should add plain old print statements in there. Data::Dumper is handy too.

    Make sure the pipe is created successfully.
    Make sure that the program is actually reaching both branches.
    See what it wants to print into the pipe.
    See what if anything it gets out. Keep in mind that it will want to slurp the whole pipe with that read.
    And make sure you're not suffering from buffering!

[reply]

      I had print statements in there. That's how I know it's getting to the write line. I've printed out $content and it's just normal old text. I can write other text into the pipe and it works. Only when I try to write $content does it hang.

[reply]

Back to Seekers of Perl Wisdom