I think there is a significant amount of time being spent...

Don't "think there is", profile, and know there is. Something as coarse grained as logging the start / stop times of each of the steps (see Time::HiRes) may be enough.