As mentioned above, you have various levels of buffering going on.

The application may buffer data before passing it to the kernel, in order to minimise the number of write system calls it needs to do. In perl, you control this with $|, in C you'd use fflush or similar.

The kernel will buffer data above the block device, so that it can try and do larger I/O requests (and also reorder I/O requests more efficiently). You can force a flush to disk with fsync (as noted above, you can get to this from perl too. fdatasync can be a better option for some scenarios, they probably don't affect you if you're writing to a raw partition.

If you have a snazzy disk controller, that may also have a write buffer. Most disks will also normally have a write buffer. 

$ sudo hdparm -i /dev/sda | grep -i writecache
 AdvancedPM=yes: disabled (255) WriteCache=enabled

[download]
So even after you've sync'd from the kernel to the hardware, your data may not yet be persistent (i.e. survive a power failure)..

You can probably use hdparm to turn off write cacheing (it might depend on your disk and/or controller - don't know). However, you may decide that this isn't desirable (filesystems don't normally operate in this mode, for example). Since your filesystem needs to be resistant to disk corruption anyway (disks can go bad) it may be worth the risk that the data doesn't arrive to the platters in the event of power failure.

Another alternative at the high-end is to buy a disk controller with a battery-backed write cache, to get the best of both world, RAM-speed writes-to-cache (subsequently batched out to magentic storage optimally), together with persistence in the event of power failure.

Perl is a fine language for many applications, but I wouldn't choose it to write a filesystem. Good luck, though :-)

In reply to Re: Need writes to happen immediately. by jbert
in thread Need writes to happen immediately. by exodist

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.