in reply to Re: Sudden problems with LWP::UserAgent
in thread Sudden problems with LWP::UserAgent

I did not try google bot. But I did try RobotUA of LWP and that didnt help either. I will investigate further and update once successful.

Thanks Uday

  • Comment on Re^2: Sudden problems with LWP::UserAgent

Replies are listed 'Best First'.
Re^3: Sudden problems with LWP::UserAgent
by udaybhaskar (Initiate) on Nov 01, 2010 at 10:26 UTC
    I got it working again. Looks like they now expect
    'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/* +;q=0.8',

    as a header field. I however am plugging in all the http headers that firefox uses for my pages of interest so that I don't end in this trouble again.

    Thanks and Regards
    Uday
      thanks a lot :-) .. you saved my time and life ... Uday !!!