I didn't set out to find this. I just happened to pass an substr ref into an existing subroutine that takes a scalar ref argument and uses substr to iterate over the data pointed at by that reference in chunks.

It was only when that subroutine that would normally take a couple of seconds to run, was still running 40 minutes later that I went looking for the reason. The effect was so dramatic that I thought it worth looking at.

The subroutine in question is one that takes large genome sequences and prints them in FASTA format (wrapped every 50 chars). The input scalars can be 100s of megabytes in length, hence the reason for passing a reference rather than a copy. Breaking such large scalars into a list using split or unpack would dramatically increase memory usage, hence the reason for iterating over the scalar using substr.

The subroutine has worked well for several years, but on this occasion I was just generating (simulating) a very large FASTA file for testing some ideas. So, instead of generating a new large scalar of random ACGTs for each record, I generated a single random sequence of the largest size possible, and was then selecting a substring of that for output as each record. It was when passing those substring refs to the FASTA output routine that I encountered the slowness.

As I was trying to generate a 3.5GB FASTA test file, something that under normal circumstances might take a couple of hours, with this combination of factors and the 2000x slowdown, it would have taken almost 6 months!