hurricup has asked for the wisdom of the Perl Monks concerning the following question:

Hi there!

Recently I've installed Apache 2.2 with worker mpm and there are additional settings for mod_perl. But I couldn't find anywhere scoping explanation.

If I start 4 servers with 64 ThreadsPerChild for example, as far as i understand, they are working pretty independently and Apache::DBI, for example, works per client process and shares cache between threads of a client, but not between processes.

The most important question is about PerlInterp* settings, like PerlInterpMax. Is it max perl interpreters per client, or limitation of total number of interpreters in all client processes?

Am I right, that perl interpreters being created and cached in the client proces and couldn't be tossed between them?

And one more question, can anyone say something about mod_perl for apache 2.4? There is no stable version and my FreeBsd hasn't started without additional shamanistic rituals.

  • Comment on mpm-worker + mod_perl2 settings meaning

Replies are listed 'Best First'.
Re: mpm-worker + mod_perl2 settings meaning
by wrog (Friar) on Nov 11, 2014 at 00:34 UTC
    Am I right, that perl interpreters being created and cached in the client proces and couldn't be tossed between them?

    Short answer: no.

    PerlInterpMax is max for the server. They are kept in a pool and assigned to child threads as needed, i.e., if there is a worker thread that needs a perl interpreter (not all of them do) for a particular request, then one gets fished out of the pool, connected up, used to service the request, and then put back in the pool when the child is finished with it. Different interpreters can be assigned to the same child at various times and a given interpreter can be used in multiple children; you have no way of knowing other than that the thread IDs can keep changing.

    This was my motivation for creating Thread::IID, which gives you intepreter IDs, because there exist applications where you need to know which interpreter is which.

    Although, to be fair, I later came to the conclusion that hardly anybody actually uses MPM-Worker on mod-perl2 production sites because of the numerous perl modules lurking out there (in 2014, still) that are not actually thread-safe (though I'll also note that it is actually possible to run MPM-Worker — I still do — you just have to be careful about which XS modules you're going to depend on; many of the core ones that matter have been taken care of at this point)

Re: mpm-worker + mod_perl2 settings meaning
by Anonymous Monk on Nov 11, 2014 at 00:43 UTC

    But I couldn't find anywhere scoping explanation.

    Let see  [ddg://site:perl.apache.org scoping], [google://site:perl.apache.org scoping] site:perl.apache.org scoping, site:perl.apache.org scoping -> http://perl.apache.org/docs/general/perl_reference/perl_reference.html

    If I start 4 servers with 64 ThreadsPerChild for example, as far as i understand, they are working pretty independently and Apache::DBI, for example, works per client process and shares cache between threads of a client, but not between processes.

    Um, https://httpd.apache.org/docs/2.2/mod/mpm_common.html#threadsperchild

    This directive sets the number of threads created by each child process. The child creates these threads at startup and never creates more. If using an MPM like mpm_winnt, where there is only one child process, this number should be high enough to handle the entire load of the server. If using an MPM like worker, where there are multiple child processes, the total number of threads should be high enough to handle the common load on the server.

    The default value for ThreadsPerChild is 64 when used with mpm_winnt and 25 when used with the others.

    And um, threads site:perl.apache.org -> https://perl.apache.org/docs/2.0/user/intro/overview.html#Threads_Support

    In order to adapt to the Apache 2.0 threads architecture (for threaded MPMs), mod_perl 2.0 needs to use thread-safe Perl interpreters, also known as "ithreads" (Interpreter Threads). This mechanism can be enabled at compile time and ensures that each Perl interpreter uses its private PerlInterpreter structure for storing its symbol tables, stacks and other Perl runtime mechanisms. When this separation is engaged any number of threads in the same process can safely perform concurrent callbacks into Perl. This of course requires each thread to have its own PerlInterpreter object, or at least that each instance is only accessed by one thread at any given time.

    The first mod_perl generation has only a single PerlInterpreter, which is constructed by the parent process, then inherited across the forks to child processes. mod_perl 2.0 has a configurable number of PerlInterpreters and two classes of interpreters, parent and clone. A parent is like that in mod_perl 1.0, where the main interpreter created at startup time compiles any pre-loaded Perl code. A clone is created from the parent using the Perl API perl_clone() (http://perldoc.perl.org/perlapi.html#Cloning-an-interpreter) function. At request time, parent interpreters are only used for making more clones, as the clones are the interpreters which actually handle requests. Care is taken by Perl to copy only mutable data, which means that no runtime locking is required and read-only data such as the syntax tree is shared from the parent, which should reduce the overall mod_perl memory footprint.

    Rather than create a PerlInterpreter per-thread by default, mod_perl creates a pool of interpreters. The pool mechanism helps cut down memory usage a great deal. As already mentioned, the syntax tree is shared between all cloned interpreters. If your server is serving more than mod_perl requests, having a smaller number of PerlInterpreters than the number of threads will clearly cut down on memory usage. Finally and perhaps the biggest win is memory re-use: as calls are made into Perl subroutines, memory allocations are made for variables when they are used for the first time. Subsequent use of variables may allocate more memory, e.g. if a scalar variable needs to hold a longer string than it did before, or an array has new elements added. As an optimization, Perl hangs onto these allocations, even though their values "go out of scope". mod_perl 2.0 has a much better control over which PerlInterpreters are used for incoming requests. The interpreters are stored in two linked lists, one for available interpreters and another for busy ones. When needed to handle a request, one interpreter is taken from the head of the available list and put back into the head of the same list when done. This means if for example you have 10 interpreters configured to be cloned at startup time, but no more than 5 are ever used concurrently, those 5 continue to reuse Perl's allocations, while the other 5 remain much smaller, but ready to go if the need arises.

    Various attributes of the pools are configurable using threads mode specific directives.

    The interpreters pool mechanism has been abstracted into an API known as "tipool", Thread Item Pool. This pool can be used to manage any data structure, in which you wish to have a smaller number than the number of configured threads. For example a replacement for Apache::DBI based on the tipool will allow to reuse database connections between multiple threads of the same process.

    So, um? Ok ;)

    The most important question is about PerlInterp* settings, like PerlInterpMax. Is it max perl interpreters per client, or limitation of total number of interpreters in all client processes?

    site:perl.apache.org PerlInterpMax -> https://perl.apache.org/docs/2.0/user/config/config.html#C_PerlInterpMax_, https://perl.apache.org/docs/2.0/user/design/design.html#item_PerlInterpMax

    If all running interpreters are in use, mod_perl will clone new interpreters to handle the request, up until this number of interpreters is reached. when PerlInterpMax is reached, mod_perl will block (via COND_WAIT()) until one becomes available (signaled via COND_SIGNAL()). Default value: 5

    Am I right, that perl interpreters being created and cached in the client proces and couldn't be tossed between them?

    Its not something I'd worry about (not a question I understand, but I think I understand that the answer isn't important, or that as a mod_perl user the answer isn't important to me )

    And one more question, can anyone say something about mod_perl for apache 2.4? There is no stable version and my FreeBsd hasn't started without additional shamanistic rituals.

    I can say nothing

    support for Apache 2.4 | ModPerl | ModPerl says

    The Debian project is shipping mod_perl with httpd24 support based on http://svn.apache.org/repos/asf/perl/modperl/branches/httpd24/.

     

     

    I don't know things but I have quotes -- Anonymous Monk