Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: Perl threading stability?

by dave_the_m (Monsignor)
on Jul 25, 2005 at 17:27 UTC ( #477912=note: print w/replies, xml ) Need Help??


in reply to Perl threading stability?

I am not aware of any problems that that a threaded build of perl causes for non-threaded applications, apart from a minor drop in performance.

As for writing threaded apps, bear in mind the following:

  • use the latest possible perl, ie 5.8.7; user-level threads only appeared in 5.8.0, and there have been many fixes in the minor releases that followed.
  • Perl ithreads are generally less efficient than forking; fork() uses the OS's memory-management facilities to do copy-on-write of the process's pages in hardware, while perl's threads->new() goes through the entire interpreter's data structures and copies each individual variable etc (only code is shared, not data).
  • Shared data in perl's ithreads is very expensive, both in terms of speed and memory usage. A shared scalar variable usually has a copy in every thread that uses it, plus a separate shared copy. When you assign to a scalar, the thread's copy is updated, then that data is also copied to the hidden shared copy. When you read from a shared scalar, the data from the hidden shared copy is copied to the thread's one. Thus if you have N threads all accessing a long string variable, your memory usage will tend towards (N+1) times the length of the string.
  • Shared arrays and hashes use a mechanism similar to tieing, which means that they are slow, but not too memory-inefficient.
  • Try to avoid having complex, deeply-nested shared data structures

Dave.

Replies are listed 'Best First'.
Re^2: Perl threading stability?
by guice (Scribe) on Jul 25, 2005 at 17:59 UTC
    Now this part surpises me:
    Shared data in perl's ithreads is very expensive, both in terms of speed and memory usage. A shared scalar variable usually has a copy in every thread that uses it, plus a separate shared copy. When you assign to a scalar, the thread's copy is updated, then that data is also copied to the hidden shared copy. When you read from a shared scalar, the data from the hidden shared copy is copied to the thread's one. Thus if you have N threads all accessing a long string variable, your memory usage will tend towards (N+1) times the length of the string.

    I would have thought Perl use references for everything, not complete copies, making things much less memory and resource intensive...

    Try to avoid having complex, deeply-nested shared data structures

    Doh! One of the ideas was to use an XML::Simple object to store server data and each thread would update that object based on possible changes ... looks like I might have to keep with fork() with a slight restructure in how i would modify that main hash.

    -- philip
    We put the 'K' in kwality!

Re^2: Perl threading stability?
by guice (Scribe) on Jul 26, 2005 at 20:56 UTC
    <blockquot>I am not aware of any problems that that a threaded build of perl causes for non-threaded applications, apart from a minor drop in performance.

    Not sure if you'd know, but I thought I'd ask; know how minor? or how impacting is it to currently non-threaded scripts?

    One thing I do have to watch out about is this version will be put out onto approximately 350 servers. At this time, my data collection scripts, only, will be using it. The collections script is not to be threaded. The threading is more for the "server" side script which gathers all the data collection dumps and loads it into the database.

    -- philip
    We put the 'K' in kwality!

      I knocked up this random meaningless benchmark script:
      my $t; for my $i (1..10_000_000) { if ($i % 3) { $t += $i; } }
      and ran it a few times on a threaded and non-threaded perl build (a recent bleedperl) and got average timings of 4.46s and 4.99s, so about 10% slower.

      Dave.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://477912]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (6)
As of 2022-10-04 14:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My preferred way to holiday/vacation is:











    Results (17 votes). Check out past polls.

    Notices?