Well, I think I kind of shot myself in the genetalia on this one. I had a working PHP script of a page that would login->parse->act->report within a Yahoo site. I wanted to make a Perl version, so got crackin'. Went all fine and dandy, until at some point today, they decided to up the security on me, forcing an anti-phishing image after I login.

I had a similar problem last night and as it turned out, I wasn't sending the proper Referer. Well, I am now, and I still get it. In fact, I tried logging in with Firefox and turning the Referer off (in Firefox) and I can log in no problem. So, they're detecing it some other way. Also note that the PHP script broke at this time, so they've basically blocked my IP, and likely any ID I've used from this IP.

Is there any other trick with Mechanize/Curl I'm missing? I'm heading to bed, but any nuggets you can offer while I sleep will be appreciated. This is driving me nuts. EDIT: It seems to be a combination of Referer + lack of cookie. If I wipe out my cookies and turn send referer off in Firefox, I'm foced w/ the image. If I delete the cookies but turn referring on, I'm not given the imge. So perhaps I'm not sending referrer as I thought I was. The other option is of course, to leave a cookie there.

In reply to WWW::Mechanize... some general scraping help by GaijinPunch

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.