in reply to Re: LWP::UserAgent Bad and Forbidden requests
in thread LWP::UserAgent Bad and Forbidden requests

Hi Anonymous,

In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround. LWP::UserAgent is kind of a grey area, since you could use it to build interactive, as well as automated web applications. However, the folks at sciencedirect.com are clearly taking a hard line against it. If taioma chooses to spoof his user agent from a spider, and it is detected, he could get himself (and his whole company) banned.

Cheers,

Larry

  • Comment on Re^2: LWP::UserAgent Bad and Forbidden requests

Replies are listed 'Best First'.
Re^3: LWP::UserAgent Bad and Forbidden requests
by tobyink (Canon) on Dec 15, 2011 at 21:06 UTC

    I agree that spoofing another user agent is an ugly hack, however setting an appropriate user agent is always a good idea:

    my $ua = LWP::UserAgent->new(agent => 'MyApp/0.1');
Re^3: LWP::UserAgent Bad and Forbidden requests
by Anonymous Monk on Dec 15, 2011 at 22:11 UTC

    In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround.

    says who?

    and his whole company) banned.

    I am not a lawyer, I don't give legal advice