In Use more threads., BrowserUk wrote:
In the long term, a better solution would be for threads::create() to accept an extra (named?) parameter that allowed the Perl programmer to specify the stack reservation on a per thread basis.
I'm working on breaking out threads into a separate CPAN module. (Just as was done with threads::shared.) Therefore, now would be a good time to consider making this sort of a change.

I'd like some ideas on what would be a reasonable syntax for named arguments.

# Possibly named argument syntax: threads->create('stack' => $size, 'code' => 'function', # or sub{} or \&func 'args' => ...); # Or using a hash ref as suggested in PBP: threads->create( { 'stack' => $size, 'code' => 'function', # or sub{} or \&func 'args' => ... } );
In the above, if the value for 'args' is an array ref, its contents would be ('unrolled' and) passed to the function as a list. Any other scalar would be passed as is.

Of course, the old syntax of threads->create(FUNC, ARGS...) would be available for backward compatibility.

Update: The more I think about it, the more questions come to me. renodino proposed using a couple of class functions to get/set a 'global' default for the stack size.

With these included in the equation, creating a thread from the 'main' thread without using the 'stack' parameter would use whatever was previously set for the default. However, what about a child thread created from a thread that used 'stack'?

# Set the 'default' thread stack size threads->set_stack_size(1_000_000); # Create a thread with a bigger stack my $child = threads->create('stack' => 2_000_000, 'code' => 'child_thread', 'args' => ...); sub child_thread { my @args = @_; ... # What's my stack size? my $grandchild = threads->create('code' => 'grandchild_thread', 'args' => ...); ... } sub grandchild_thread { my @args = @_; ... }
Does the $grandchild thread inherit its stack size from its parent ($child = 2 million) or does it use the 'global' default (1 million)?

Remember: There's always one more bug.

Replies are listed 'Best First'.
Re: New threads->create() syntax
by BrowserUk (Patriarch) on Feb 28, 2006 at 22:46 UTC

    I think that renodino's idea of a separate api for setting the default stacksize is a good one. I'd store the value into the PerlIterpreter struct (interp.h) with a default value similar to the current stacksize setting for compatibility. Once altered, and new threads would inherit the new size. If it gets reset in a spawned thread, any threads spawned from that thread would inherit the new value.

    That does mean your separate package would have a dependancy upon the addition of that new field to the core, but I don't see why that would be a show stopper.

    I'd also add an extended threads->create() method, (maybe threads->createEx() if that's not 'too MS'?), that took a hashref as the first parameter and the standard args thereafter. The contents of the hashref would be a set of (potentially platform specific), named args that would be passed either directly or in modified form to the CreateThread()/pthreads_create() calls. If the hashref was empty (or undef is passed) then it maps to the normal create() method.

    The keynames could be mapped to the standard parameter names used by the platform api docs, giving you a pretty extensible api that would allow for future expansion if the need arises. For example, I could see the possibility of passing a SECURITY_ATTRIBUTES structure being a useful facility.

    While you're there :), could you also add an instance method for accessing the native thread handle from the Perl thread object $threadObject->getNative()?

    That would facilitate using platform specific apis (like SetThreadPriority(), suspend/resumeThread(), pthreads_suspend_np() etc.), from add-on packages without requiring the main threads package to support everything for all platforms.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: New threads->create() syntax
by renodino (Curate) on Feb 28, 2006 at 21:46 UTC
    Whatever the anser to your update questions, I hope that the new create() signature is in addition to support for set/get_stack_size() ?

    I don't have any complaint wrt the named param signature (in fact I'd suggest possibly providing support for priority params if possible - tho implementation of that capability is very hit-and-miss, and usually platform specific), but set_stack_size() permits existing apps to just add a single call up front, wo/ otherwise touching existing code, to benefit from smaller stack size.

    As for your child/grandchild conundrum, I'd say inheritance is probably the best rule to fall back on, since its likely the desired behavior for any other attributes (e.g., priority). I suppose you'd need to differentiate between a parent thread that had explicitly set/inherited an attribute vs. using the global value, in the event the global value is modified at some point between parent and child creation.

Re: New threads->create() syntax (best practices)
by tye (Sage) on Mar 01, 2006 at 07:40 UTC

    This sounds like a job for a few of my Perl OO 'best practices'.

    First, use a hash ref not a list of pairs pretending to be a hash for the new API. It makes distinguishing between the old and new API clearer to both the module user and the module's code. It also leaves the most room for subsequent improvements to the API [ such as allowing ->create( {stackBytes=>32*1024}, subName, @args ) ]. Also, it makes the code generally clearer, including putting things like 'odd number of elements' warrnings where they make more sense.

    Second, self-factories. Class-level configuration variables suck. But it is even easier (down-right trivial) to implement great "cascading configuration (defaults)" in Perl OO modules.

    For module configuration parameters, the first thing you need is a good set of defaults. Then you need a place to store User A's default configuration choices that might be different from User B's defaults. Then you need to know what configuration to use for creating this next thread, merging the global defaults with the user's overrides to these defaults, with the options given in the call to create().

    But, like I said, this is actually trivial to do and even gives the user the ability to have multiple layers of defaults and provides other API benefits.

    Just store the parameters in a simple object, such as a hash. Then your global defaults simply become a class-global hash declaration. Users get their own identical object as a 'factory' where they can set whatever defaults they want and then create threads from the configured factory. Then they can make this factory as global or as limitted in scope as their heart desires.

    And the object that gets associated with the new thread [the return value from ->create(...)] can be the exact same type of object, just with more data added in.

    So it is as simple as:

    my $default= { stackBytes => 4*1024, ..., }; sub getDefault { return $default; } sub new { my $us= shift @_; my $opts= shift @_; if( $opts && ! isa($opts,"HASH") ) { $opts= { code => $opts, args => \@_, }; } my $class= ref($us) || $us; my $proto= ref($us) ? $us : $class->getDefault(); my $self= bless { }, $class; my @configKeys= keys %{ $class->getDefault() }; # add error checking to taste here, # but this conveys the idea @$self{ @configKeys }= @$proto{ @configKeys }; @$self{ keys %$opts }= values %$opts; return $self; }

    Then the user can say

    my $threads= threads->new( { stackBytes => 16*1024, # default ... } ); ... my $worker= $threads->create( ... ); my $deep= $threads->create( { stackBytes => 32*1024 }, ..., ); $threads->setStackBytes( ... ); my @hive= map $threads->create( ... ), ...;

    [ Note that new() and create() will be nearly identical and can be completely identical, depending on your API design tastes. ]

    Now, if this were Perl 6, then the module user would probably get their first factory via something like:

    my $threadFactory= use threads;

    which has some nice advantages (like not requiring you to repeat the module name all over, the benefits of which are less obvious in the case of 'threads.pm', but mostly do still apply).

    In Perl 5, the best you can do is a little bit more awkward. Now, the above example code means that people can go very traditional like:

    require threads; ... my $child= threads->create( \&mySsub );

    Or set up their personal defaults:

    require threads; my $threads= threads->new( { stackBytes => ..., } );

    But I like to make that easier, like:

    use threads( Factory => \my $threads, StackBytes => ..., );

    Which is pretty easy to do with a simple import() sub.

    [ I notice that I've violated my own advice by not passing a hash ref to use threads and I'm unsure whether 'use' is enough of a special case that this departure is worthwhile... ]

    - tye        

Re: New threads->create() syntax
by renodino (Curate) on Mar 01, 2006 at 18:56 UTC
    One other thought:

    threads has been a part of CORE for awhile now, which means there's at least some legacy apps out there (including a number of my own) that might want to take advantage of the stack sizing.

    Alas, sysadmins and production environments being what they are, installing modified code - no matter how minor the change - is nearly impossible, or at least a herculean effort.

    So I'll suggest an additional i/f: a PERL5_ITHREADS_STACK_SIZE environment variable that can be applied globally to enable the stack trimming wo/ any code mods, just install the new threads package from CPAN, set the environment variable, test the app as needed, and install into production. Just add water, makes its own sauce...

    While this sort of interface might be frowned upon aesthetically and for long term support, I'd suggest we're dealing with some seriously low hanging fruit here for getting ithreads footprint under size 20 EEEE (at least on some commonly used platforms), if the production deployment impact can be minimized.

Re: New threads->create() syntax
by radiantmatrix (Parson) on Mar 01, 2006 at 21:32 UTC

    My thoughts, some of which may be redundant:

    1. Set stack size at threads object construction:
      my $thr = threads->new(stack_size => 65535);
    2. Or set the default during import:
      use threads { default_stack => 65535 };
    3. Or set stack size for a given thread:
      $thr->create({stack=>65535}, 'function', 'arg1','arg2');
      If the first option is a HashRef, treat it as options to create(); if it's a scalar, it's the function name (old-style call).
    4. Or, instead of the above, use a separate method:
      $thr->create('function','arg1','arg2'); # use defaults $thr->create_with( -call => 'function', -args => ['arg1','arg2'], -stack => 65535 );
    5. And still allow runtime changes to (and discovery of) settigs:
      my $stack = $thr->get_stack_size() + 1024; $thr->set_stack_size($stack) if $stack <= 2048;

    I know I mixed naming conventions for the stack size parameters; I did this to suggest several alternatives, and not to suggest that the chosen convention should be anything but consistent.

    <-radiant.matrix->
    A collection of thoughts and links from the minds of geeks
    The Code that can be seen is not the true Code
    I haven't found a problem yet that can't be solved by a well-placed trebuchet