Now this part surpises me:
Shared data in perl's ithreads is very expensive, both in terms of speed and memory usage. A shared scalar variable usually has a copy in every thread that uses it, plus a separate shared copy. When you assign to a scalar, the thread's copy is updated, then that data is also copied to the hidden shared copy. When you read from a shared scalar, the data from the hidden shared copy is copied to the thread's one. Thus if you have N threads all accessing a long string variable, your memory usage will tend towards (N+1) times the length of the string.
I would have thought Perl use references for everything, not complete copies, making things much less memory and resource intensive...
Try to avoid having complex, deeply-nested shared data structures
Doh! One of the ideas was to use an XML::Simple object to store server data and each thread would update that object based on possible changes ... looks like I might have to keep with fork() with a slight restructure in how i would modify that main hash.
-- philip
We put the 'K' in kwality!
| [reply] |
<blockquot>I am not aware of any problems that that a threaded build of perl causes for non-threaded applications, apart from a minor drop in performance.
Not sure if you'd know, but I thought I'd ask; know how minor? or how impacting is it to currently non-threaded scripts?
One thing I do have to watch out about is this version will be put out onto approximately 350 servers. At this time, my data collection scripts, only, will be using it. The collections script is not to be threaded. The threading is more for the "server" side script which gathers all the data collection dumps and loads it into the database.
-- philip
We put the 'K' in kwality!
| [reply] |
I knocked up this random meaningless benchmark script:
my $t;
for my $i (1..10_000_000) {
if ($i % 3) {
$t += $i;
}
}
and ran it a few times on a threaded and non-threaded perl build (a recent bleedperl) and got average timings of 4.46s and 4.99s, so about 10% slower.
Dave. | [reply] [d/l] |