Hi monks

I'm going to keep this rather general. I've got WWW::Scripter working which slurps a few pages for me. There's some rather nasty Javascript on at least the first page which is detecting the client (and throwing a captcha out if it doesn't like you). Scripter alleviates this, but with a big caveat: performance is really slow. On my remote server this can take a good 60 seconds to parse through this script. The script itself is about 165k so it is not trivial. Clearly a parser is the right answer, rather than attempting to figure out what it does and reimpliment the logic in the script..

Does this sound like I'm doing something wrong, or is this expected behavior? Any of the following pages detect if Javascript is enabled (via running a javascript), so my one idea of using Scripter for the first page and simply sharing the cookie with Mechanize for subsequent pages won't work. I've tried using both the Javascript & Ajax plugins. They seem to offer me about the same in terms of speed.

Any pointers, or even a "yeah, Scripter can be slow" would be appreciated.

In reply to WWW::Scripter performance by GaijinPunch

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.