I've been doing some analysis of some of my perl scripts... Take this simple example reading/writing a 1 meg file (with about 3000 lines):- 
#!/usr/bin/perl
use Fcntl qw(:DEFAULT :flock);
$|=1;
print "Content-type:text/html;charset=ISO-8859-1\n\n";

open DF,"test.txt";
@test=<DF>;
close DF;

my $rec;
foreach(@test){$rec.=$_;}
sysopen (DF,"test.txt",O_WRONLY | O_CREAT);
syswrite DF,$rec,length($rec);
close DF;

exit;

[download]
When I watch this running (XP Pro SP2) using File Monitor (by SysInternals) I can see the read generates a new IO process 4K at a time. Worse still, when writing, it generates an IO process for each 1 K... In total this simple operation generates over 1500 IO processes (in File Monitor).

Using standard open and print results in the same number of IO processes (in File Monitor). I've also tried the same thing running perl in CYGWin with File Monitor looking, and the same results are shown, 4K chunks are read, and 1K chunks are written.


Is there anyway around this? Am I monitoring it correctly? You can see by my example I've tried using the more exotic calls to try and stop this "buffering"...

Here's a link to an example output from FileMonitor showing the 4K chunks being read in. www.hvweb.co.uk/fawcettn/filemon4.gif (Rememember to maximise the image size)

UPDATE---- Seems when I test using CYGWin BINMOD makes NO improvement at all... So what right? I want to improve the software on my ISPs Unix machine, but get contradictory results from Active Perl and CYGWin on my XP system... What can I do :(

In reply to Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO! by NeilF

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.