in reply to Why would LWP::Simple::get stop working?

It could be that the server in question started checking for the user agent type of all incoming requests, and rejecting those corresponding to the $ua object of LWP::Simple. To test this hypothesis, add the following line somewhere after the definition of my $ua, but before calling LWP::Simple::get:

$LWP::Simple::ua->agent( $ua->agent );
The idea is to make the agent field LWP::Simple's internal user agent object (which is also called $ua) match the corresponding default agent field for a LWP::UserAgent object (like your $ua).

Update: I should have pointed out in my original post that if the hypothesis above is correct, this would strongly suggests that the site you are trying to access has an anti-robot policy. The workaround implied by the code above is a technical solution, but it is up to you to determine what the site's robot policy is (starting with reading their robots.txt file, if any), and weigh the consequences of circunventing it. Some site's are very strict and unequivocal about this.

Update 2: As indicated by the overstrike above, the particular details of what I proposed were wrong, because importing $ua from LWP::Simple has side-effects that don't take place if one omits the import step and uses the fully qualified $LWP::Simple::us, which I was not aware of (thanks++ to dpmott for pointing it out).

the lowliest monk

Replies are listed 'Best First'.
Re^2: Why would LWP::Simple::get stop working?
by dpmott (Scribe) on Jun 20, 2005 at 20:05 UTC
    This suggestion is getting me closer.

    It turns out that you must have a 'use' statement that looks like this to get access to the $ua object:
    use LWP::Simple qw/$ua/;
    If you want to use get() without fully qualifying it, then you also have to include that along with $ua.

    When I specify the '$ua' in the import list, everything starts working, and working well.

    If I do not specify anything in the import list, then LWP::Simple::get() or just get() never returns.

    It looks like my PPM is broken, too, so I think that I have a re-install in my near future. I must've gotten my hands on a bad module somewhere...