in reply to Webbot with Perl for Factiva data collection

greenjr:

If the page submits the searches as standardized form, then you could probably just create and submit the forms without worrying about interpreting JavaScript stuff. (By standardized form I mean that it always uses the same variable names that you can fill in.)

Of course, if it's an AJAX-y app, all bets are off...

...roboticus

  • Comment on Re: Webbot with Perl for Factiva data collection

Replies are listed 'Best First'.
Re^2: Webbot with Perl for Factiva data collection
by greenjr (Initiate) on Apr 06, 2007 at 13:19 UTC
    That's what I thought I was trying to do, but I guess I am not sure how to fill in the variable names...
[reply]
      greenjr:

      Use a tool (ethereal, etc.) to capture the POST (or whatever) using a web browser. Then you can simply generate a similar POST, filling in the blanks with your new values. Then parse the resulting page for the information you desire. (Insert standard warning about the unsuitability of regular expressions for parsing HTML here.)

      ...roboticus

[reply]
        I'll work on that and see where I get, Thanks
[reply]

In Section Seekers of Perl Wisdom