Re^2: collect data from web pages and insert into mysql

Thanks!

It's nice and refreshing to be greeted in such a friendly and helpful way. I think I have Mechanize now (did the "cpan WWW::Mechanize" from prompt and lots of stuff happened ;) ).

I guess the first part as training will be reading the pid list from a file, setting it as variable and then create a file with pid as name and the repeat untill the list is done.
(more or less the start and end of the entire project).

Comment on Re^2: collect data from web pages and insert into mysql

Replies are listed 'Best First'.
Re^3: collect data from web pages and insert into mysql by SteinerKD (Acolyte) on Jul 30, 2010 at 22:28 UTC
Hmm, actually managed (with the help from AWP) to create a valid URL (inserting pid and page number) for a sortie list page and have the source printed to screen. Looking at the resulting code it should be simple (I think) to create a list of sids (sortie pages) to process as they are all listed in the source as sid=XXXXXX (inside a string). Must say my head is spinning a bit though, this is a lot to take in (I mainly copied something I found and adapted it, not like I could write it from scratch myself). I guess next step would be to store the page as a temp file and figure out how to grab and save those sids. Thanks for encouragement and help!	[reply]

Replies are listed 'Best First'.

Re^3: collect data from web pages and insert into mysql
by SteinerKD (Acolyte) on Jul 30, 2010 at 22:28 UTC

Hmm, actually managed (with the help from AWP) to create a valid URL (inserting pid and page number) for a sortie list page and have the source printed to screen.

Looking at the resulting code it should be simple (I think) to create a list of sids (sortie pages) to process as they are all listed in the source as sid=XXXXXX (inside a string).

Must say my head is spinning a bit though, this is a lot to take in (I mainly copied something I found and adapted it, not like I could write it from scratch myself).

I guess next step would be to store the page as a temp file and figure out how to grab and save those sids.

Thanks for encouragement and help!

[reply]