This is a two part question.

I want to submit to 9,000 forms (all across one domain) and it's not spamming, just so you know.

Question 1: How do I LOG IN and STORE THE COOKIES so I can access these 9,000 pages? I know some of you will post links to read a FAQ or something but I've tried doing this a number of times in the past with separate scripts and never have I successfully been able to scrape pages OR submit data to pages that required a login first.

Question 2: Since this is 9,000 pages and I don't know fork(), there's no way this as a CGI script could execute without timing out. I don't think the script would be THAT fast!

So this question is, I think breaking it into three scripts would be best. 1 login form which sends data to the other two scripts. One script submits the data to the first half of 9,000 pages and the second would submit to the other half of 9,000. The only problem with this is I wouldn't be able to see live updates as to which page it's up to if it's two scripts.

So any ideas on how this would work? It doesn't necessarily have to print out a message for each submission, maybe for each 10 or 100 just so you know it's still running.

Any help with the cookies and ideas with question #2 would be very helpful!

In reply to automated form processing with cookies by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.