The problem

We have a reasonably large collection of packages (over 100). We are starting to run into problems with load times and more specifically, the amount of code that needs to be compiled and loaded into memory. The issue stems from the cascade of use statements. Script A uses package B which in turn uses package C which ends up through a huge chain of dependencies 'use'ing 50% of our packages. We end up with a situation where the login page ends up including a module that is entirely unrelated to the login process because one of the modules needed for login needs the unrelated package for a few of its routines.

I am fully aware that this is a hole that we dug for ourselves and that the real answer is to structure our package dependencies in such a way that this doesn't happen. That said, at this point in time the cost to go through and restructure 100k+ lines of code is prohibitive. We are going to be making baby steps to resolve these issues as we have time, but we aren't going to be able to put everything on hold while we fix it.

The idea for a fix

An idea that we've been toying with is to use UNIVERSAL::AUTOLOAD to deal with loading the appropriate packages at runtime. Through some inspection of some of the problem packages it turns out that many of the included packages are included for a function that only gets used in a few specifc places. If those functions aren't called we don't need to load the package.

The important thing to note is we are running CGI's. They load, run, then exit. There is no obvious win by having everything loaded up front. We don't use export very often (for our in house written packages) and we don't use OO functionality in many places either. The other thing to note is that we tend to use fully qualified function names Foo::Bar::func() so we'd be able to reliably figure out what package needs to be used.

The question

We are wondering if there is anything inherantly broken with this approach. At the end of the day the real answer is to re-structure the code to avoid this problem. However, we're looking for something to fix some of our issues before we are able to entirely fix it.

In reply to Dealing with large chains of dependent packages by amw1

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.