in reply to Optimizing perl code performance

Remove the processing and replace it with just a print DATAOUT $line;. Time this. See how much time is actually spent in the processing part of this. I would guess that the largest part of your time is spent doing I/O, especially if $infile and $outfile are on the same data path (controller, disk, ...).

Once you show that a significant portion of time is spent in the processing, and that splitting the work will help, _then_ perhaps you _might_ gain value from splitting the processing apart into multiple workers. Personally, I would guess from experience that you will get better performance gains by putting your input and output files on different data paths.

--MidLifeXis

Replies are listed 'Best First'.
Re^2: Optimizing perl code performance
by BillKSmith (Monsignor) on Aug 15, 2015 at 01:03 UTC
    The standard advice for optimizing any code is to profile it, and then optimize only those parts which are shown to consume a large part of the processing time. In this case, MidLifeXis's suggestion accomplishes much the same thing with much less work.
    Bill