in reply to Re: Performance and CPU load: sysread, IO::Select and/or Storable::thaw
in thread Performance and CPU load: sysread, IO::Select and/or Storable::thaw

Your reply makes perfect sense. The parent runs for quite a long time and starts only under very specific circumstances. It's also working with some legacy logic, so it's hard to test. I have, however, modified the child to remove the 5 second time out from can_read(5) because the IO::Select docs say:

If "TIMEOUT" is not given and any handles are registered then the call will block.

I also added the line:

my ($handle) = $select->can_read(5); $handle->blocking(1);

This had no effect either, presumably because the handle to \*STDIN had already been created. I also tried setting that handle to blocking(1) before any loops start, which also had no effect:

my @handles = $select->handles(); $handles[0]->blocking(1);

I'm concerned that even if I set the read handle to block in the parent when open2() is called it still won't work. Perhaps I need both the write handle and read handle to block?

Forgive my IPC ignorance here, but what I'm trying does not seem to match what I've read and I tried to do an extremely thorough job of research and testing.

Replies are listed 'Best First'.
Re^3: Performance and CPU load: sysread, IO::Select and/or Storable::thaw
by jethro (Monsignor) on Jun 29, 2010 at 18:21 UTC

    I would have uncommented any calls to $handle->blocking(0). The default mode is blocking, so that should work, and work even after forks.

    Naturally that could make other problems if some other part of the code relies on non-blocking reads, but it would be only for testing the hypothesis

    The timeout of 5 seconds is quite ok. If the loop only runs every 5 seconds you will never notice it. What the docs mean is that if you don't specify a TIMEOUT value, the timeout value will be infinity. But 5 seconds is practically infinity, if you look at cpu load

    So uncomment or remove the blocking(0) calls, add print lines around your can_read calls to check if they wait for 5 seconds. If they still don't wait, check how many bytes they read out of the socket, maybe whatever code that writes to the socket is broken and spams the socket with data.

      In theory, your comments make perfect sense and your advice is great. I've actually set up a full test environment for these and tried every variation of blocking() including commenting it out as you suggest. I've also tried replacing can_read() with IO::Select::select().

      In every single case, the IO::Handle object returns 1 for blocking(), but can_read returns immediately, regardless of timeout value. I am almost 100% certain the parent is not sending any more data than the message that needs to be thawed. No matter what, CPU spikes to 100% in the child until it dies. The only conditions that mitigate this are when I reduce the timeout on can_read() or select and even then CPU is between 60-80% for each child. Improved, but still a resource drain.

      I'm very perplexed as to why can_read() and select will not block in this application.

        I looked a bit closer at your code and noticed that you add STDIN to your select call instead of readHandle. Any reason? It might be that this is your problem.

        I tried the following test script and it works. I get output every 5 seconds:

        #!/usr/bin/perl use warnings; use Data::Dumper; use IO::Handle; use IPC::Open2; use IO::Select; my $i; my ($readHandle, $writeHandle) = (IO::Handle->new, IO::Handle->new); my $pid = open2($readHandle, $writeHandle, 'bc'); $writeHandle->autoflush(1); $readHandle->autoflush(1); my $select = IO::Select->new(); $select->add($readHandle); while (1) { print $i++,"\n"; my ($handle) = $select->can_read(5); }

        Interestingly it works even when I turn off blocking, which means my previous notion that can_read would not wait if blocking is turned off might be false.