You may be able to use fork to speed up the process. Whether it actually does is hard to say - measure it. Since you need to feed information back, I would have the parent parse the file; for each matching line, it forks a child - but not using fork directly, but using a pipe-open. Fork children until you've reached the maximum number of children. Then use a select loop over the pipes to see which child is done processing - if a child is done, read its results, process them, then parse the file for the next match. Rinse and repeat until you went through the entire file, and all children have finished.