Controlling Shared Memory Id's

rkac has asked for the wisdom of the Perl Monks concerning the following question:

I am currently working on a rehat6.2 intel platform and have had some interesting problems. A data base program that we run uses up shmid's every time we run the cli. I currently can run approximately 30 instances from my perl program but I start getting "shared memory device unavailable" errors if I attempt to run more. I noticed that alot of the shared memory ID's seemed to have 0 processes connected to them and that eventually they would dissappear but not very quickly. I added this as a shared memory controller to remove id's that had 0 attached and my performance increased dramatically and the number of instances could go higher.

 
while (1 == 1)
{


     
    #
    #Clear any unused shared memory pages
    #
    

        $presult=`ipcs -m |grep -v - | grep -v bytes | grep -v dest`;
        @parts = split("\n", $presult);
        foreach $w (@parts)
        {
        print "ipcs: $w\n";
            @pidparts=split(" ", $w);
                $thispid = $pidparts[1];
                $presult1=$pidparts[5];   
                if ($presult1 eq  0)
                {
                    print "removing ssh id: pid: -$thispid- current at
+tached -$presult1-\n";
                        $pcmd="ipcrm shm  \"$thispid\"";
            print "pcmd: $pcmd\n";           
                        $presult1 = `$pcmd`;
            print "presult1: $presult1\n";
            sleep 15;
                }
                         
        }        

    
}
[download]

Unfortunately, what it also does ocassionally is kill shared memory id's while they are being initialized (not all, just some). Is there any way to read the contents of the shared memory page, using perl, to see what is happening and based on that knowledge choose to remove it?

Comment on Controlling Shared Memory Id's Download Code

Replies are listed 'Best First'.
Re (tilly) 1: Controlling Shared Memory Id's by tilly (Archbishop) on Sep 14, 2000 at 02:11 UTC
Unfortunately you have a race condition to deal with here. What I would suggest as a first solution is to use some sort of semaphore or other locking to force your Perl programs to not be in the process of initializing memory devices at the moment you clear. Failing that being convenient, have your shared memory device note which devices have zero processes, wait, then only kill the ones which still do. As long as that wait is significantly longer than the initialization time, you should be fine. Incidentally your controller should probably go a little more slowly. Right now when it has nothing to do you wait in a tight loop and it probably generates an appreciable load...	[reply]
Re: Controlling Shared Memory Id's by rkac (Novice) on Sep 14, 2000 at 02:39 UTC
Thanks Tilly, Good plans of attack... I like it. The Perl executes a commercial .exe which I cannot modify so keeping a record of initialization time would be best for me. The control loop added 1-2 to the machines load. I thought this was caused by more instances of the command line interface. I thought the performance benefit of quickly removing supposedly unused shm id's was worth the trade off.	[reply]
RE (tilly) 2: Controlling Shared Memory Id's by tilly (Archbishop) on Sep 14, 2000 at 02:46 UTC
The machine's load is the average number of processes that were ready to run when the scheduler went to allocate a time slice. That will rise 1 because either your script or the shells it launches can always run. And you may add a bit more because other processes will spend time waiting that they would have spent running. Unless these instances are being launched at a crazy speed, you will probably find sleeping 1 second in the loop to be well worthwhile. Even if they are launched fast, tossing in a `select(undef,undef,undef,0.05)` will still drop your load substantially and get rid of them quickly.	[reply]
Re: Controlling Shared Memory Id's by rkac (Novice) on Sep 15, 2000 at 23:30 UTC
Here's the final code. Any comments appreciated. while (1 == 1) { # #Clear any unused shared memory pages # print $x++ ."\n"; %oldpids=%newpids; %newpids=(); select (undef,undef,undef,1.25); $presult=`ipcs -m \|grep -v - \| grep -v bytes \| grep -v dest`; @parts = split("\n", $presult); foreach $w (@parts) { select (undef,undef,undef,0.05) ; #print "ipcs: $w\n"; @pidparts=split(" ", $w); $thispid = $pidparts[1]; $presult1=$pidparts[5]; if ($presult1 eq 0) { $newpids {$thispid} = $thispid; print "checking ssh id: pid: -$thispid- current attached - +$presult1-\n"; select (undef,undef,undef,0.05) ; if ($oldpids{$thispid} == $thispid) { print "removing ssh id: pid: -$thispid- current attach +ed -$presult1-\n"; $pcmd="ipcrm shm \"$thispid\""; print "pcmd: $pcmd\n"; $presult1 = `$pcmd`; print "presult1: $presult1\n"; select (undef,undef,undef,0.05) ; } } } } [download]	[reply] [d/l]