File I/O is (almost) completely dependent on the speed of your hard-drives. A 7200rpm disk is going to be half as fast as a 15000rpm disk. Now, depending on the setup of your disk(s), that can be altered. For example, if you use certain RAID operations, that can significantly speed up your reads, but slow down your writes. Additionally, depending on which filesystem you use, that can have an impact. For example, ReiserFS is much better at many small files than ext2 or ext3. Journalling filesystems are generally slow with writes than non-journalling filesystems, for example ext2 is faster than ext3. (If you need journalling, you're not going to comlpain.)

Overall, however, opening and closing files is generally O(1), which means they generally are rather fast. Now, reading and writing data can be more costly. But, the cost is usually in the memory structures people read into than in the actual cost of reading. Writing, especially appending as in a logfile is almost O(0); in other words, it's so negligible as to be irrelevant.

Now, my question is why are you closing the file after every iteration? It sounds like you really only want to flush the buffers after every iteration. So, you could do something like: 

use IO::File;

my $fh = IO::File->new( ">>$filename" ) or die "Cannot open '$filename
+' for appending: $!\n";

# Do your stuff here. At the end of every iteration, call $fh->flush;

$fh->close;

[download]

Now, an alternative would be to turn autoflushing on, using $|++; at the top of your script, preferably in a BEGIN block. There are many developers, including merlyn and Abigail-II that do that reflexively. I didn't suggest that at first for a few reasons:

  1. That turns autoflushing on for EVERY filehandle. You might not want that.
  2. Buffering is the default for a reason - the act of writing generally costs as much for 30bytes as for 30K. So, you might as well write as few times as possible.
  3. This turns off buffering for reads as well. You might not want that.
  4. It sounds like you want to control when you flush your buffers. Autoflushing takes that control away from you.

You just have to figure out what's best for your specific need.

Being right, does not endow the right to be rude; politeness costs nothing.
Being unknowing, is not the same as being stupid.
Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

In reply to Re: Intelligent logging by dragonchild
in thread Intelligent logging by Muoyo

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.