in reply to Re^3: Isolating dynamically loaded modules with Thread::Isolate.
in thread Isolating dynamically loaded modules with Thread::Isolate.

I you don't need inter-task communications, then I agree, but the biggest problem people have with using threads is trying to use them in the same way that forks are used. This is true of whether using iThreads or pthreads, or most other forms of threading.

Frankly, my biggest problem with using threads in Perl is that the implementation isn't very good. It's slow, uses a ton of memory and has some very non-intuitive restrictions. It feels exactly like what it is - a second-rate feature added as an after-thought.

-sam

  • Comment on Re^4: Isolating dynamically loaded modules with Thread::Isolate.

Replies are listed 'Best First'.
Re^5: Isolating dynamically loaded modules with Thread::Isolate.
by BrowserUk (Patriarch) on Jan 31, 2005 at 04:32 UTC
    It's slow, uses a ton of memory

    The following code spawns 100 threads in under 2 seconds and uses < 40MB? 

    >perl -Mthreads -le"print time;@t=map{threads->new(sub{sleep 300})}1..
+100;print time;<>"
1107144770
1107144772

    [download]

    I agree they aren't doing much, and that is probably considerably slower than you can spawn 100 processes--but if you need the parent to communicate with the children and vice versa, then you'd need to a lot more work using fork.

    It is true that there are some inconvenient restrictions (some of which could be alleviated), and you do have to learn a few simple techniques for getting the best from them--like not loading anything you don't want shared before you spawn your threads--but they are stable and very usable.

    On Win32, they are infinitely preferable to fork, but I realise that that is a restriction of the platform. I can well see why anyone on unix wouldn't bother with them unless they really needed bi-directional communications.

    I agree that the implementation is imperfect, but having spent a little time looking into the work that they are doign, I think teh guys that got them to where they are are did an brillaint job given the task they faced.

    Examine what is said, not who speaks.
    Silence betokens consent.
    Love the truth but pardon error.
[reply]
[d/l]
      I suspect 100 forked processes that don't load any modules would use a lot less than 40MB, and it only gets worse for threads when you start loading enough modules to do something useful. The code and variable storage used by the modules will all be duplicated into every thread. With forking, copy-on-write prevents this. Forking is not great, but at this point it's a lot lighter than threads.
[reply]
        The code and variable storage used by the modules will all be duplicated into every thread.

        Not so, if you defer the loading of the modules and do it only within the thread or threads where it is required. That is what my post was demonstrating.

        By requireing those modules required by each thread from within that thread, only that thread gets that code and data. If the thread then terminates, the memory used by those things it loaded is returned to the process' memory pool. Indeed, it is entirely possiible to have a single process that has HTTP::Daemon loaded in one thread; LWP in another; CGI in another; with none of them duplicated.

        It is the fork-like "replicate everything currently loaded" behaviour that creates the problem. As I said elsewhere, if the choice of what was duplicated was left entirely to the programmer--life would be much simpler and iThreads would be considerably faster to start and lighter in use.

        With forking, copy-on-write prevents this

        If your OS supports fork-with-COW, then you're laughing--provided you do not need for the separate parts of your overall task to communicate. Under those circumstances, I too would use fork, and I do advocate that.

        Examine what is said, not who speaks.
        Silence betokens consent.
        Love the truth but pardon error.
[reply]

In Section Meditations