bennierounder has asked for the wisdom of the Perl Monks concerning the following question:

Hi guys I am wanting to build a bot place to bets for me at certain times on Betfair, using perl. I will program the bot to do things in certain situations. This is perfectly legal because betfair allow it's site users to make and use bots.

I am wanting to find out how to scan a website and return information into a data structure within perl (scalar,array,hash, etc). I am guessing this will be using CGI.PM, but then again I'm not sure. Does anyone know where I can find some literature I can study to accomplish the above task or tell me things I need to accomplish it. I just need to know what to study. I have been told you can scan a website for data, but have no idea how it's done. any help will be much appreciated. Thank you

Replies are listed 'Best First'.
Re: Perl Online Bot
by Corion (Patriarch) on Jun 28, 2012 at 12:14 UTC

    If Betfair allows bots, most likely they also have some documents about how they expect bots to talk to their computers. Before trying to automate a browser and simulate a user, I would look at what direct methods they offer, as usually these are far more convenient ways than to extract information intended for humans from pages that may change.

    If, on the other hand, you do not want your programs to be recognized as programs, you can try to simulate a human (or a browser, potentially controlled by a human) by using a module in the Mechanize family of programs, or Web::Magic, Web::Scraper or Mojolicious. The term you might want to search for is "web scraping" or "screen scraping".

      You guys are amazing, thanks for the info

Re: Perl Online Bot
by moritz (Cardinal) on Jun 28, 2012 at 12:13 UTC