exodist has asked for the wisdom of the Perl Monks concerning the following question:

I am writing a perl program that directly writes data on a raw partition, it is mission critical that it is written in order and at the time I write it, no buffering..

I am using sysopen, sysread, sysseek to do this.
I have set HANDLE->autoflush(1)
and I have set $| = 1;
It is obviously still buffering however as it completes writing 100 mb in 1024 bytes at a time alternating between the end and beggining of a 25gb partition in 2 seconds! HD's are bad for random access liek this, whereas flash drives are supposed to be better, it takes the flash drive 36 seconds for the same thing to occur!

Replies are listed 'Best First'.
Re: Need writes to happen immediately.
by GrandFather (Saint) on Jun 17, 2007 at 00:45 UTC

    There is no seek time overhead with flash (as there is with a hard drive), however writes take a relatively long time and may occur in blocks as do writes to a hard drive. Flash is essentially memory speeds for random access read and something like hard drive speed (or slower) for writes with similar constraints over needing to write fixed size blocks rather than write random bytes.


    DWIM is Perl's answer to Gödel
      I know the speed differences and the advantage / disadvantage of each. My problem is that I cannot seem to find a way to directly access the devices and bypass all the caching and everything that occurs, because of this I cannot do what I need to and make sure writes occur in order at the time they are given.

      I am at my wits end, I do not know where to go from here... My searches have shown that someone doing something similar needed to use mmap to map some memory and align it with something before using O_DIRECT. but the terminology is beyond me. I try using O_DIRECT and get an invalid argument.

      Now I am trying it with O_SYNC but all it is doing is making it write at about 1mb every 2 minutes.
        Perhaps it would help if I just explained why it is so important.
        I am writing a filesystem, I am using perl to quickly prototype it and write something that works, I will then likely try to make a kernel moduel (though my c/c++ skills are not very good, and the kernel code is way over my head)
        The key is That the database that keeps track of where files are on the disk, and the journal that records operations and weather or not they finished has a critical need to be written in the order I specify. For instance if I am copying a file to the device the journal NEEDS to have it written that the operation was started, then the file is written and I do not care about caching, I will flush when it is done. Then I NEEd the journal to have an entree saying the operation finished. It is critical that the journal operations happen when I specify.
        Since flash drives have faster random access I am using them to store the database while I use regular disks to hold the actual files since they are larger and continuous.
        Before anyone tries to point out that it may have been done before or where I want to say I do not care, I am not doing this to be original or to get any kind of recognition, I am doing it to learn how to do it.
Re: Need writes to happen immediately.
by varian (Chaplain) on Jun 17, 2007 at 10:43 UTC
    Sysread and syswrite use system calls read/write and unbuffered I/O so that's a start. Now you may still be impacted by firmware and system software optimalization routines that will attempt to combines read/write operation such that a minimum of disk head movements is required.... And yes there is a solution to use syscall to call the fsync operating system function to ensure that your data is sufficiently committed to the device. In reality that will slow down the overall disk operations even further.

    it is mission critical that it is written in order and at the time I write it
    If you really want to have full control over the device itself, then you need to write a device driver that will run within the kernel.

    Now if this is as you state a learning experiment and you mean to be able to influence sequence of events then you may want to dive into the world of semaphores and use the flock function to perform mutually exclusive operations. This would closely mimick some of the internal system operations that you work on.

Re: Need writes to happen immediately.
by jdporter (Paladin) on Jun 17, 2007 at 16:54 UTC

    Some caching happens that the OS level, so the answer to your question may/will depend on what OS you're using. If it's Windows, see How to Manually Enable/Disable Disk Write Caching. I've tried this for flash drives and it works just fine.

    A word spoken in Mind will reach its own level, in the objective world, by its own weight
Re: Need writes to happen immediately.
by jbert (Priest) on Jun 18, 2007 at 16:00 UTC
    $| controls application-level buffering. It ensures that perl will call write() with your data.

    In general, the kernel is under no obligation to push this data to the underlying storage in anything like a timely fashion. You want fsync() for this job, which - as someone else points out - you can get to via syscall from perl.

    If what you want is a sync after every write, then O_SYNC is pretty much that. You mention that this is very slow - so maybe check (with strace) that you are writing in the right-sized chunks, rather than character-at-a-time.

Re: Need writes to happen immediately.
by jbert (Priest) on Nov 27, 2007 at 10:24 UTC
    As mentioned above, you have various levels of buffering going on.

    The application may buffer data before passing it to the kernel, in order to minimise the number of write system calls it needs to do. In perl, you control this with $|, in C you'd use fflush or similar.

    The kernel will buffer data above the block device, so that it can try and do larger I/O requests (and also reorder I/O requests more efficiently). You can force a flush to disk with fsync (as noted above, you can get to this from perl too. fdatasync can be a better option for some scenarios, they probably don't affect you if you're writing to a raw partition.

    If you have a snazzy disk controller, that may also have a write buffer. Most disks will also normally have a write buffer.

    $ sudo hdparm -i /dev/sda | grep -i writecache AdvancedPM=yes: disabled (255) WriteCache=enabled
    So even after you've sync'd from the kernel to the hardware, your data may not yet be persistent (i.e. survive a power failure)..

    You can probably use hdparm to turn off write cacheing (it might depend on your disk and/or controller - don't know). However, you may decide that this isn't desirable (filesystems don't normally operate in this mode, for example). Since your filesystem needs to be resistant to disk corruption anyway (disks can go bad) it may be worth the risk that the data doesn't arrive to the platters in the event of power failure.

    Another alternative at the high-end is to buy a disk controller with a battery-backed write cache, to get the best of both world, RAM-speed writes-to-cache (subsequently batched out to magentic storage optimally), together with persistence in the event of power failure.

    Perl is a fine language for many applications, but I wouldn't choose it to write a filesystem. Good luck, though :-)

Re: Need writes to happen immediately.
by fishnuts (Acolyte) on Nov 26, 2007 at 20:53 UTC
    I'm having a similar problem, except with sysread, which this mmap-facilitated buffer alignment method isn't working for. Has anyone used this method to use sysread successfully on an O_DIRECT filehandle?

    It's at node 653075.