Its a pretty common problem on linux, with Perl threads. The best solution I have come up with, is to reuse the threads. You set a max number of threads to be handled at any one time, then feed them data to process thru threads:shared variables, and when they are done, don't try to destroy them; instead, clean out their old data, and reuse them, as if they are in a circular worker queue. See Reusable threads demo

That way, your memory will increase for say 20 worker threads, then will stay relatively constant.

It seems reasonable programming style to reuse massive thread objects, rather than destroying and recreating them. Remember, filehandles can be shared across threads, thru the fileno.