Re: LWP::UserAgent Bad and Forbidden requests

(my browser displays them without a problem):

pretend to be your browser

This is the solution to every such problem, see Re^5: can't get WWW::Mechanize to sign in on JustAnswer or http://www.perl.com/pub/2004/06/04/recorder.html#Web Testing with HTTP::Recorder or WWW::Mechanize::Firefox

Comment on Re: LWP::UserAgent Bad and Forbidden requests

Replies are listed 'Best First'.
Re^2: LWP::UserAgent Bad and Forbidden requests by 1arryb (Acolyte) on Dec 15, 2011 at 20:07 UTC
Hi Anonymous, In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround. LWP::UserAgent is kind of a grey area, since you could use it to build interactive, as well as automated web applications. However, the folks at sciencedirect.com are clearly taking a hard line against it. If taioma chooses to spoof his user agent from a spider, and it is detected, he could get himself (and his whole company) banned. Cheers, Larry	[reply]
Re^3: LWP::UserAgent Bad and Forbidden requests by tobyink (Canon) on Dec 15, 2011 at 21:06 UTC
I agree that spoofing another user agent is an ugly hack, however setting an appropriate user agent is always a good idea: `my $ua = LWP::UserAgent->new(agent => 'MyApp/0.1');` [download]	[reply] [d/l]
Re^3: LWP::UserAgent Bad and Forbidden requests by Anonymous Monk on Dec 15, 2011 at 22:11 UTC
In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround. says who? and his whole company) banned. I am not a lawyer, I don't give legal advice	[reply]