Re: collect data from web pages and insert into mysql

I'm not sure that I understand the "pid" and "sid" stuff, but the answer is yes, this kind of task (retrieve a web page, parse for data, store the retrieved data into a DB) can be done with Perl. For the web pages, you can use a module like LWP to grab the contents. From there, you can either use regular expressions to manually parse the data or you might find an HTML parser module that may help out. As for the DB connection, you'll probably want to use DBD::mysql, which will allow you to query your MySQL DB as well as do inserts and updates.

Since you're new to Perl, you'll definitely want to keep http://search.cpan.org handy. That's where you'll be able to search for other Perl modules.

Welcome to the wonderful world of Perl!

Comment on Re: collect data from web pages and insert into mysql

Replies are listed 'Best First'.
Re^2: collect data from web pages and insert into mysql by SteinerKD (Acolyte) on Jul 30, 2010 at 15:57 UTC
The pid and sid I mention as just 6 digit numbers identifying personas and sorties. What I'm going to do is collect stats for a squad I'm part of for a online game for starts and reward purposes. Each member have 3 different personas (pids) and those are inserted into URLs to get a page with their lists of sorties. Each line contain a link to a sortie page (url+sid, again sig digit number). Those pages contains the main data, some basic data and two lists with either kills or captures (stored in table row blocks) Does that makes things clearer? Thanks for the suggestions, I'll look those up, but will probably be quite a while before I get even close to anything useful.	[reply]
Re^3: collect data from web pages and insert into mysql by dasgar (Priest) on Jul 30, 2010 at 16:41 UTC
Now that you've mentioned going to one web page and following a link to another (I must have missed that point when I first read your post), I'd agree with "Your Mother"'s suggestion of using WWW::Mechanize. Also, since you mentioned that this is to gather player stats from an online game, you might want to search for that game's name at CPAN. There's a chance that someone may have written a module to do some/all of the work you're wanting to do. My philosophy is: why reinvent the wheel if you don't have to?	[reply]