Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Using Time::HiRes, requireing CGI.pm takes 3.39746475219727e-05 seconds. requireing my own file (which is comprised of ~15 packages in a single file) takes 0.121029019355774 seconds. My file is about a third of the size of CGI.pm, yet takes much much longer to load.

What could my problem be?

Replies are listed 'Best First'.
Re: Require Load Times
by IlyaM (Parson) on Dec 30, 2001 at 03:07 UTC
    Probably because most of CGI subroutines are autoloaded. If you look into sources of CGI.pm you can find a code that looks like:
    %SUBS = ( ... ... ... 'some_sub1' => <<'END_OF_FUNC', sub some_sub1 { ... ... } END_OF_FUNC 'some_sub2' => <<'END_OF_FUNC', sub some_sub2 { ... ... } END_OF_FUNC ... ... ... )
    As you can see defenitions of subroutines are put into a hash as strings. When Perl parses such code it does it very quickly since it doesn't have to parse actual subroutine code.

    CGI.pm uses AUTOLOAD to detect undefined subroutine calls and to evaluate subroutines code on fly when they are used.

    Update: You can use autoloading techiques yourself for your modules. Probably you don't need to roll your own code for autoloading like CGI.pm does. Use one of these modules: AutoLoader, SelfLoader and autouse.

    --
    Ilya Martynov (http://martynov.org/)

      That's kind of an interesting way of doing it. I wonder how CGI.pm's method of doing things compares to using autoloader? Typically, whenever I create a module, I prefer to just use h2xs -X modulename to set up my Makefile.pl and stub module. That stub module is already set up to use Autoloader. Would I be better off speed wise if I tried to implement my modules the same way that CGI.pm does?

      In my particular case, most of my work is done with mod_perl, so I'm guessing that using Autoloader is probably better from a memory efficency perspective. Then again, I suppose CGI.pm is typically used in mod_perl as well. Can anybody shed some light on this subject?

        The short answer is that it depends. The long answer is that shared memory is very very good in mod_perl situations. Some people go as far as to force CGI to compile all of the common subroutines when the server starts -- that way, they'll be available to any child that needs them in a shared memory page. If they had to be autoloaded occasionally during a request, each child would have its own copy in a fresh memory page.

        The mod_perl guide has lots of suggestions and equations.

Re: Require Load Times
by demerphq (Chancellor) on Dec 30, 2001 at 04:43 UTC
    As IlyaM says its because CGI is AUTOLOADing his subs on demand. But my question is this: Why do you care? A 10th of a second is hardly a long time to wait, and if your code does any IO at all that time will get completely swamped.

    But if it really is an issue there are examples of handrolled autoloads in perltoot and perltootc, in addition to the prepacks that IlyaM mentions.

    Oh and iirc there are issues with autloading methods, something about spoiling your method cache. But i cant remember where I heard that.

    Yves / DeMerphq
    --
    When to use Prototypes?

      There is indeed a method cache issue.

      Whenever you call a method, Perl has to figure out what that method is. While the search rules are clear, the search can be slow. So what Perl does is assumes that when you call a method once, you are likely to call it again, and so it stores in an internal hash the information about where the method is so that next time the lookup will be fast.

      Unfortunately if you change any information which could change the lookup results, Perl has to throw away this cache and start over because it can no longer trust the answers. Normally this isn't much of a problem because people tend to use everything, define all of their functions, and then start calling methods (which builds the cache). So once you start running your code, your cache never gets thrown away again.

      Unfortunately dynamic changing of functions at runtime (which is what autoloading them does) means that in the middle of running your code you are forced to throw away your cache. Do this repeatedly and you spend a lot of startup time building up bits of this cache only to throw it away again and start over.

      If your process runs for a long time, or if you can arrange to preload everything that you are going to use up front, this is not an issue.

        Ok, I guessed it was something like this, but hoped/expected that the method cache wouldn't get blown when a method was added to the cache and not changed.

        But I can see how an all or nothing strategy would be easier to deal with.

        Thanks Tilly.

        Yves / DeMerphq
        --
        When to use Prototypes?