Hi
I am also planning to forcibly make it to sleep for a few seconds after 50,000 files have gone through , will this add a tinge to make it a non crashable program.
This not the solution. This does nothing other than sleeping and again wasting your time instead of your CPU time.
Probabaly what you would require, is to change your algorithm if possible. Else change the way you read your files. ie read it in blocks so u would have little memory usage.
Show us your code that could help us in determining an optimal solution.
Also try to visit this page
http://www.ccl4.org/~nick/P/Fast_Enough/ which can help you. Also dont forget see this node
422918 This has some expert comments that might help you