in reply to Re^3: Performance and CPU load: sysread, IO::Select and/or Storable::thaw
in thread Performance and CPU load: sysread, IO::Select and/or Storable::thaw

In theory, your comments make perfect sense and your advice is great. I've actually set up a full test environment for these and tried every variation of blocking() including commenting it out as you suggest. I've also tried replacing can_read() with IO::Select::select().

In every single case, the IO::Handle object returns 1 for blocking(), but can_read returns immediately, regardless of timeout value. I am almost 100% certain the parent is not sending any more data than the message that needs to be thawed. No matter what, CPU spikes to 100% in the child until it dies. The only conditions that mitigate this are when I reduce the timeout on can_read() or select and even then CPU is between 60-80% for each child. Improved, but still a resource drain.

I'm very perplexed as to why can_read() and select will not block in this application.

  • Comment on Re^4: Performance and CPU load: sysread, IO::Select and/or Storable::thaw

Replies are listed 'Best First'.
Re^5: Performance and CPU load: sysread, IO::Select and/or Storable::thaw
by jethro (Monsignor) on Jun 30, 2010 at 09:47 UTC

    I looked a bit closer at your code and noticed that you add STDIN to your select call instead of readHandle. Any reason? It might be that this is your problem.

    I tried the following test script and it works. I get output every 5 seconds:

    #!/usr/bin/perl use warnings; use Data::Dumper; use IO::Handle; use IPC::Open2; use IO::Select; my $i; my ($readHandle, $writeHandle) = (IO::Handle->new, IO::Handle->new); my $pid = open2($readHandle, $writeHandle, 'bc'); $writeHandle->autoflush(1); $readHandle->autoflush(1); my $select = IO::Select->new(); $select->add($readHandle); while (1) { print $i++,"\n"; my ($handle) = $select->can_read(5); }

    Interestingly it works even when I turn off blocking, which means my previous notion that can_read would not wait if blocking is turned off might be false.

      The $readHandle is defined in the parent, not the child. The child is reading from STDIN, so I don't believe this will work, and in my testing, it did not.

      I do, however, have a workaround. I set the timeout to 0 based on your advice and put a very small delay at the end of the "infinite" loop:

      while(time < $endTime) { [...] select(undef, undef, undef, 0.05); }

      Since I could not get blocking to work no matter what I tried (which may still be some fault of my own), this seems to be a workable fix for now. It reduced CPU to almost nothing, which is an enormous gain since the load was consistently over 80 on the box where this was running multiple instances.

      I will continue to modify my test code to see if I can implement your suggestions more effectively. Your help has been invaluable in at least finding a workaround and possibly a permanent solution, jethro, and I really appreciate your time