Re^2: Connection Timeout duing form submissions

dates.txt looks fine. No whitespace or incorrect numbers around 18961110.

And as far as leeching the site for the files, this is a university site for the college I attend and have been instructed to gather this info by my advising instructor. The faculty is aware that such projects are taking place. The real question is why does the program keep failing at 1896110?

Comment on Re^2: Connection Timeout duing form submissions

Replies are listed 'Best First'.
Re^3: Connection Timeout duing form submissions by Marshall (Canon) on Jun 20, 2009 at 22:53 UTC
Ok. I will suggest this again, run your program for some dates like August 1, 1921 to December 23, 1922. I think also that you should be "polite" regarding number of hits per second on the other website. The previous poster suggested this and I agree. Get your script working on a limited date range. Then expand that date range. Get your data and then "shut up". I would put some "sleep()" into the script and just let it run for a day. The data from 1920 isn't going to change. For your school project the objective shouldn't be: how to get this data as fast as possible, it should just be: how do I get this data? I also haven't yet seen any "this is what was sent" (the actual stuff) vs "this is what I received". I haven't seen any boundary test cases based upon what you have heard so far.	[reply]

Replies are listed 'Best First'.

Re^3: Connection Timeout duing form submissions
by Marshall (Canon) on Jun 20, 2009 at 22:53 UTC

I think also that you should be "polite" regarding number of hits per second on the other website. The previous poster suggested this and I agree.

Get your script working on a limited date range. Then expand that date range. Get your data and then "shut up". I would put some "sleep()" into the script and just let it run for a day. The data from 1920 isn't going to change. For your school project the objective shouldn't be: how to get this data as fast as possible, it should just be: how do I get this data?

I also haven't yet seen any "this is what was sent" (the actual stuff) vs "this is what I received". I haven't seen any boundary test cases based upon what you have heard so far.

[reply]