in reply to Splitting large module

G'day jimmygoogle,

Welcome to the Monastery.

We really need to see your split code as well as your unsplit code to make a comparison.

I could suggest looking at the parent pragma:

package A; ... use parent 'Foo'; ...

But, of course, you may be using that already.

Here's some things you could do to improve your post and get better help from us:

— Ken

Replies are listed 'Best First'.
Re^2: Splitting large module
by jimmygoogle (Initiate) on Jul 11, 2018 at 13:02 UTC

    Pardon my noob-ness. Let me add some more details and try to answer some questions. The code is running as a mod_perl app so I would think all of this code would be loaded when the server is started up. I do need all of them loaded since I dont know when they might be used. Unfortunately use of use parent isnt that easy in my ecosystem. We have development environments on 5.8 still and other environments on 5.20.2. That wrinkle aside, I am stuck with this hardware and configuration, I need to make due with what I am given.

    So the unsplit file looks a little more like this.

    package ThisIsMyPackge; package Foo; package A our @ISA = qw{Foo}; sub validate { do something; } package B our @ISA = qw{Foo}; sub validate { do something; } package C our @ISA = qw{Foo}; sub validate { do something; } package Bar; package D our @ISA = qw{Bar} sub apply { do something; } package E our @ISA = qw{Bar} sub apply { do something; } package F our @ISA = qw{Bar} sub apply { do something; }

    And the split version now looks like this:

    package ThisIsMyPackge; use A; use B; use C; use D; use E; use F; use 79 more times ... .....

    I didnt use Benchmark, I used Time::HiRes to calculate the time it takes to run the foreach loop (modified) below. The timings are based on average of 5 runs through the code. I have done more runs but the results dont differ so I use 5 for my calculations. Note this foreach loop isnt called in ThisIsMyPackge.pm, its called from another file.

    unsplit: .14858s

    split: .4153s

    foreach my $bar (@{$objects}) { my $foo = $bar->object; .... next unless $foo->validate; $bar->apply; .... }

    I put it in other timings to isolate the bottleneck and the slow down is around the $foo->validate and $bar->apply calls. All of the other logic in the loop has its timings almost exactly the same. The results from Devel::NYTProf also point to this area of the code as well. This might not seem like a lot of time but over the course of thousands of concurrent hits, it is a pretty big decrease in performance.

      "Pardon my noob-ness."

      No need to apologise. That was your first post.

      "The code is running as a mod_perl app ..."

      It's been over a decade since I did any serious work with mod_perl; I doubt I'm qualified to give much in the way of advice. I do seem to recall some sort of pre-load (or maybe pre-fetch, or something like that) feature: perhaps look into that.

      "I do need all of them loaded since I dont know when they might be used."

      Does that also equate to "don't know if they might be used"? You might consider loading modules on demand. ++hippo discussed this. Loading a core set of modules initially, then others only when needed, would at least spread the load time, even if you do eventually want all of them.

      "We have development environments ... I need to make due with what I am given."

      Yes, I understand that; I've been in that situation myself. If you do end up sticking with your current unsplit setup, I'd just recommend fully documenting what you have and making that documentation obvious. There's certainly been times when I've looked for "lib/X/Y/Z.pm" and, after much frustrated searching, found "package X::Y::Z;" in "lib/M/N/O/P.pm".

      — Ken

        Alright so I figured it out. The file I was trying to split was ~14K lines. We were given a script that broke it up for us. It turns out that the script was not perfect. I went through by hand and split the file up and I am seeing the same performance now from both sets of files. Thanks for the input everyone.

      I need to make due with what I am given.

      It's "make do".

        It's "make do".

        Except on Windows, where it's "gmake do".