in reply to Re^9: Slow evolution of Perl = Perl is a closed Word (thread decade)
in thread Slow evolution of Perl = Perl is a closed Word

   1. $obj->startLongOrBlockingProcess()
   2. $obj->isProcessComplete()
   3. $obj->getProcessResults()

[download]

With current ithreads implementation, this solution is not as elegant as it could seem. Creating a new thread can be a very expensive operation (in terms of both CPU and memory) that depends on the in-memory size of the runtime. So you can not pretend that threading business are completely hidden behind your class API.

I have this problem when porting Net::SFTP::Foreign to Windows: I needed to do non-blocking IO on a pipe, but windows only supports select on sockets. Using threads to handle IO would be the best alternative (one thread for reading from the pipe and other for writting), but I found it unacceptable because I knew nothing about the caller. For instance it could be a program using 500MB of RAM, and starting the two new threads would triplicate that!!!

Replies are listed 'Best First'.
Re^11: Slow evolution of Perl = Perl is a closed Word (thread decade)
by BrowserUk (Patriarch) on Sep 04, 2007 at 09:04 UTC
    ... it could be a program using 500MB of RAM, and starting the two new threads would triplicate that!!!

    What? Care to explain that? Better still, care to demo it?

    I would guess that you are alluding to CLONING here? Only explicitly shared data gets cloned on thread creation, and then only if thread creation occurs after the creation of the shared data. It is avoidable, with care, though I agree that is far from ideal.

    I have been asking for a 'bare thread' NO_CLONE option for months years, but FUD like the above doesn't help.

    I needed to do non-blocking IO on a pipe, but windows only supports select on sockets.

    FYI: It is perfectly possible, and reasonably simple to do non-blocking IO on a pipe under win32 without using threads. You can't use select to do it directly, but with sufficient XS/internals skills, the nitty-gritty could be hidden under the covers of Perl's select.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
      What? Care to explain that? Better still, care to demo it?

      Sure, launch your favorite OS monitor and enjoy: 

      use strict; use warnings;
use threads;
$| = 1;
{
  package foo;
  sub run { threads->create(sub { sleep 10 }) }
}
{
  my @a;
  for (1..3e6) { push @a,  "foo".rand }
  print "memory allocated\n";
  sleep 10;
  print "now running threads\n";
  foo->run;
  foo->run;
  sleep 10;
}

      [download]
      Only explicitly shared data gets cloned on thread creation

      No, it is exactly the opposite, everything but shared data gets cloned when a thread is created.

      FYI: It is perfectly possible, and reasonably simple to do non-blocking IO on a pipe under win32 without using threads. You can't use select to do it directly, but with sufficient XS/internals skills, the nitty-gritty could be hidden under the covers of Perl's select.

      So, can you point me in the right direction?

[reply]
[d/l]
        Only explicitly shared data gets cloned on thread creation
        No, it is exactly the opposite, everything but shared data gets cloned when a thread is created.

        It is my impression that unshared items get cloned but that shared items get cloned and then a third item is created to mediate between the two clones. So slightly worse in terms of efficiency of thread creation.

        - tye        

[reply]
        No, it is exactly the opposite,...

        You're right. I had a brain fart. Too many concurrent thought trains.

        What you need is mod://threads::Bare. Don't go looking for it because it doesn't exist (yet!). I think I have a pure perl solution that will work on win32 and may be made to work on other platforms, but it would be much better and more simply coded as a part of threads if you have the appropriate skill set and thick skin.

        So, can you point me in the right direction?

        See PeekNamedPipe and the second code snippet in Re: Non-blocking Reads from Pipe Filehandle for the (incomplete) basis to non-blocking IO from win32 anonymous pipes. There is another (older?) post around somewhere with some other details I think, but I've never had cause to completely solve the riddle from Perl.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Meditations