Thanks for all the replies and suggestions! After I posted it did occur to me to write out direct to file instead of using the mergedlogs intermediate variable and that had a huge speed improvement. I ran both versions, and the original is still going after 16 hrs, whereas the improved version finished in about 30 minutes. I'll investigate the other suggestions too.