Given the OP's observations about memory patterns, I rather suspect that the sample programs aren't representative of what's really happening. It might be as simple as no "my" in

  my $str .= stuff;
[download]

or it might be something deeper. If the full code isn't available for us to diagnose, and memory use is acceptable if batch sizes are reduced, I say reduce the batch sizes and concatenate.