in reply to Benign Web Miner

Is there a reason why you are not using a DB (postgreSQL being my pick for the amount of concurrent connections you're using) for a backend? This would alleviate the disk frag problem.

If you wanted to continue using wget then create a separate partition holding the downloaded data temporarily until a writer writes to a DB would work.



grep
Mynd you, mønk bites Kan be pretti nasti...