You need to dump the library and roll your own. %files is eating up memory since you are storing the data there for a time before using it. Write a breadth first traversal routine and stream the output to the browser without storing more than one line at a time. Use a command pipe with a sorted ls command as the stream. Use counters and -d -f tests to print stats. You can probably find a breadth first traversal on the web.