http://qs1969.pair.com?node_id=477958


in reply to Re^2: Perl threading stability?
in thread Perl threading stability?

I don't buy Perl not being able to use references for it's data ...

You obviously do not appreciate the problems involved.

... since it's done in real threading applications just fine.

What do you call "real threading"? Are you thinking of Kernel threads in C or User threads in say Java?

Each of these forms of threading have their own set of demands and limitations. In addition, chosing to use either language to benefit from that flavour of threading imposes the disadvantages that language imposes upon the rest of your application.

My idea wants the ability to share data across threads while maintaining somewhat of a speed benefit from multiple children.

Please describe your application in some detail. Ithreads can be used to accomadate many uses, but it does require that you understand both their advantages and limitations.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Replies are listed 'Best First'.
Re^4: Perl threading stability?
by guice (Scribe) on Jul 25, 2005 at 21:05 UTC

    When thinking threading, I always viewed it in the eyes of user threading like in Java. Granted java is memory intensive, it still is efficient at the code level.

    Please describe your application in some detail. Ithreads can be used to accomadate many uses, but it does require that you understand both their advantages and limitations.

    Right now the application is built using forks. Each fork spawns off a DBI connection of it's own and reads in data from a text file based off the system it's currently updating within the database (data to update is gotten remotely by another script).

    The restrictions I currently have and trying to fix is data sharing. Right now when I get an error, I dump it to a temp file and then read in the temp file at the end of the processing. This is possibly one of the most efficient ways of doing it due to the size of the load I have I still rather find something a bit more "less problemantic".

    Hashes storing data like server hostnames and a hostname changes. I have a DB loaded hash containing all "active" systems. I need the ability to "change" that has in the case of a hostname change.

    I'm also wanting to build an XML file containing human entered discription data for each system (right now it uses bunches of little name=val strings--one file per server). 300+ systems, while a single file might be too much, it's definatly too much for threads. Either case, when a hostname, etc, changes on a system, I need that change updated within the XML hash as well for dumping at the end of the script execution.

    -- philip
    We put the 'K' in kwality!

      When thinking threading, I always viewed it in the eyes of user threading like in Java.

      The main limitation of java-style user threads is that there is no true concurrency. No matter how many cpus or cores the machine has, a Java app will only utilise one of them at a time no matter how many user threads it spawns. Therefore, you can never scale the app by moving it to a multi-cpu system.

      That same limitation is what allows Java threads to be very lightweight and efficient. As only one thread is ever truely running at any given moment, many of the consideration for locking required by apps utilising kernel threads simply do not arise.

      It's a swings and roundabouts argument. What you gain in one place, you loose in another.


      Your 'spec' is still pretty sketchy, but if I'm reading between the lines correctly, I would say that what you want to do is eminently doable with iThreads.

      To be sure, I would need further details including the number of systems you are talking to concurrently; the nature of the data you are sharing etc.

      Not only does it sound doable, it sounds pretty straight forward.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
        Java will use multiple CPUs for multiple threads depending on the platform and the thread library. The JVM may run multiple Java threads on top of a single kernel thread, but all the thread libraries I know about will do multiple kernel threads. My impression was that recent JVMs on Linux run directly on the pthread library and each thread is a separate kernel thread. The kernel threads on Linux are efficient enough that implementing threads in user space does not help.
        Not only does it sound doable, it sounds pretty straight forward.

        I'm pretty sure it is pretty straight forward. My only concern is taking up too much resources. The current script (using max if 5 fork()'d children) already takes ~4-5 hours to run... Albeit some things aren't at it's most efficient, I just don't want it to take longer.

        From what I'm seeing with Perl threads is that it's really CPU/Memory intensive vs general fork(). That's a lot to ask for just for the ability to share data, my primary concern. Not to mention Perl's history of being flakey with Threads (although a post below has stated it's stablalized now).

        In the future I might see about writing this in Java, but at this time it's just not feasiable due to current data imports (standard Dumper dump of variables into a file). But that's not for at least 6+ months, until I can get XML inputs and data use stablized.

        If you want to chat more about specs, feel free to email me gp-at-gpcentre.net. I don't really want to clutter this post up with "my problem" per say. More use it to discusse perl threading in general.

        -- philip
        We put the 'K' in kwality!