Re^3: Running out of memory...

I realise you have fixed your problem, but I have to say that without seeing your actual data, 5800 is a very strange choice of buffer size?

Examine what is said, not who speaks.

Silence betokens consent.

Love the truth but pardon error.

Comment on Re^3: Running out of memory...

Replies are listed 'Best First'.
Re^4: Running out of memory... by knewter (Novice) on Mar 03, 2005 at 20:22 UTC
I don't exactly remember why I did that now, but it had something to do with the average size of the html file I was slurping...that number's something like twice it I think, so that I'm guaranteed to have the data I need in the middle somewhere...yeah, I don't remember why exactly now, but I'm glad to say that the scraper portion of my project is completed and that the data is happily in a database serving out useful information to people that didn't have it as easily before :)	[reply]

Replies are listed 'Best First'.

Re^4: Running out of memory...
by knewter (Novice) on Mar 03, 2005 at 20:22 UTC

I don't exactly remember why I did that now, but it had something to do with the average size of the html file I was slurping...that number's something like twice it I think, so that I'm guaranteed to have the data I need in the middle somewhere...yeah, I don't remember why exactly now, but I'm glad to say that the scraper portion of my project is completed and that the data is happily in a database serving out useful information to people that didn't have it as easily before :)

[reply]