Re: Making an array from a downloaded web page

Also try increasing your count per page to make for fewer pages to fetch/parse. 100 items per page seems to be their maximum. http://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0000051143&owner=include&count=100

Comment on Re: Making an array from a downloaded web page

Replies are listed 'Best First'.
Re^2: Making an array from a downloaded web page by malomar66 (Acolyte) on Jan 16, 2007 at 06:13 UTC
I planned to go to 100 actually. It will be more efficient. But the need to hit that next link for more data will still be necessary.	[reply]
Re^3: Making an array from a downloaded web page by lv211 (Beadle) on Jan 16, 2007 at 20:40 UTC
You can use WWW::Mechanize's follow_link method. It might look like this depending upon the size of the query you are doing. I think it says Next 40, but you might want to do Next 100 if that's the max they allow. `$mech->follow_link( text => 'Next 40');` [download] http://search.cpan.org/~petdance/WWW-Mechanize-1.20/lib/WWW/Mechanize.pm#%24mech-%3Efollow_link(...)	[reply] [d/l]