in reply to Re^2: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!
in thread Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!

The problem is, it is quite likely that your ISP is measuring your IO in terms of bytes read and written rather than the number of reads and writes, so reducing the latter is unlikely to satisfy them.

Also, when you have read the entire file, there is no need to re-write the entire thing in order to add a new line. If you open the file for reading and writing, when you have read it, the file pointer will be perfectly placed to append any new line to the end. That will reduce your writes to 1 per new addition. If there is no new addition, they user is just refreshing, then you'll have no writes.

Also, you presumably do not redisplay the entire forum each time, but rather only the last 20 or so lines?

If this is so, then you should not bother to re-read the entire file each time, but rather use File::ReadBackwards to get just those lines you intend to display. If you do this, then you can use seekFH, 0, 2 to reposition the pointer to the eof and then append new lines without having to re-write the entire file each time.

Using this method, you can fix the total overhead per invocation to (say) 20 reads and 0 or 1 writes. You'll need to deploy locking, but from your code above you seem to be already familiar with that.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re^3: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!
  • Download Code

Replies are listed 'Best First'.
Re^4: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!
by NeilF (Sexton) on Jan 02, 2006 at 15:10 UTC
    Thanks... Interesting stuff! They do seem to talk specifically about IO processes! Is File::ReadBackwards a standard library/module?? ie: Will it exist on sites?
[reply]
      They do seem to talk specifically about IO processes!

      Hmm. As a phrase "IO processes" doesn't make a lot of sense.

      The lines you see, and the numbers in the first column under the heading "#" in Filemon, are IO events, not processes. The process IDs are appended to the filenames in the third column under the heading "Process". The best I can suggest to you is that you ask them by what measure they are deciding that you are using to much resource.

      File::ReadBackwards isn't a standard module, but it is pure perl, so it is easy to install it in the same place as your scripts live, and use use lib './lib'; to allow it to be found. Assuming your scripts live in a subdirectory /cgi-bin, the you would create a directory structure and copy Backwards.pm from CPAN into it as follows: 

      /cgi-bin/lib/File/Backwards.pm

      [download]

      Then in your script you would have 

      use lib './lib';
use File::Backwards;

      [download]

      I seem to remember someone posting a more thorough explanation of this somewhere, but I could not find it via supersearch. Maybe someone else remembers it and will post a link.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]
[reply]

In Section Seekers of Perl Wisdom