in reply to Performance and CPU load: sysread, IO::Select and/or Storable::thaw

It seems you are turning off blocking on the filehandles. I might be wrong, but if the handles don't block then "$select->can_read(5);" should not wait 5 seconds but return instantly (because as I understand it, can_read (via select()) tries a read on a filehandle and blocks until it can read or a timer runs out)

You might check that easily by putting a "print ++$x;" statement near the can_read line and see if there is a 5 second wait inbetween the numbers. Then turn on blocking reads and see if anything changes

UPDATE: Read the select() man page (http://linux.die.net/man/2/select), it says there that select returns when you can read a filehandle without blocking

  • Comment on Re: Performance and CPU load: sysread, IO::Select and/or Storable::thaw

Replies are listed 'Best First'.
Re^2: Performance and CPU load: sysread, IO::Select and/or Storable::thaw
by DBX (Pilgrim) on Jun 28, 2010 at 22:48 UTC

    Your reply makes perfect sense. The parent runs for quite a long time and starts only under very specific circumstances. It's also working with some legacy logic, so it's hard to test. I have, however, modified the child to remove the 5 second time out from can_read(5) because the IO::Select docs say:

    If "TIMEOUT" is not given and any handles are registered then the call will block.

    I also added the line:

    my ($handle) = $select->can_read(5); $handle->blocking(1);

    This had no effect either, presumably because the handle to \*STDIN had already been created. I also tried setting that handle to blocking(1) before any loops start, which also had no effect:

    my @handles = $select->handles(); $handles[0]->blocking(1);

    I'm concerned that even if I set the read handle to block in the parent when open2() is called it still won't work. Perhaps I need both the write handle and read handle to block?

    Forgive my IPC ignorance here, but what I'm trying does not seem to match what I've read and I tried to do an extremely thorough job of research and testing.

      I would have uncommented any calls to $handle->blocking(0). The default mode is blocking, so that should work, and work even after forks.

      Naturally that could make other problems if some other part of the code relies on non-blocking reads, but it would be only for testing the hypothesis

      The timeout of 5 seconds is quite ok. If the loop only runs every 5 seconds you will never notice it. What the docs mean is that if you don't specify a TIMEOUT value, the timeout value will be infinity. But 5 seconds is practically infinity, if you look at cpu load

      So uncomment or remove the blocking(0) calls, add print lines around your can_read calls to check if they wait for 5 seconds. If they still don't wait, check how many bytes they read out of the socket, maybe whatever code that writes to the socket is broken and spams the socket with data.

        In theory, your comments make perfect sense and your advice is great. I've actually set up a full test environment for these and tried every variation of blocking() including commenting it out as you suggest. I've also tried replacing can_read() with IO::Select::select().

        In every single case, the IO::Handle object returns 1 for blocking(), but can_read returns immediately, regardless of timeout value. I am almost 100% certain the parent is not sending any more data than the message that needs to be thawed. No matter what, CPU spikes to 100% in the child until it dies. The only conditions that mitigate this are when I reduce the timeout on can_read() or select and even then CPU is between 60-80% for each child. Improved, but still a resource drain.

        I'm very perplexed as to why can_read() and select will not block in this application.