Getting rid of Moose sounds like a good idea. There are lighter variations (Moo, Mouse) that may work, but perl can do OO just fine without extra modules.

Also consider using Devel::NYTProf for a much more detailed analysis of your code. Devel::DProf is deprecated, and Devel::NYTProf is the suggested replacement.

Alexander