in reply to WWW::Robots problem

Since LWP::RobotUA @ISA LWP::UserAgent, you're getting the default behavior where a 301/302 is handled "internally" for you. You'll need to instantiate your own LWP::RobotUA so that it doesn't follow the 30x, and then use the USERAGENT attribute of WWW::Robot to use your instance instead.

Something like this might work:

my $www_bot = WWW::Robot->new( ... USERAGENT => LWP::RobotUA->new(requests_redirectable => []), ... );