I'm attempting to use each of sysopen, sysseek, sysread, and syswrite to directly access a shared scsi block device. The sysopen call is using the O_DIRECT flag to accomplish unbuffered IO, but a caveat is that all reads/writes via the filehandle must be aligned to a block boundary. It's very important that all IO to/from the device be direct and un-cooked, because multiple physical machines read and write to different parts of the same device simultanously -- It's a shared state database for a high-availability cluster, and it's imperative that I get the freshest data from the device every time I read from it.

I've used sample code from node 621630 and used the technique given there using Sys::Mmap to get a block-aligned buffer. This block-aligned buffer can be used with syswrite, successfully, but sysread using the same exact buffer still fails with an 'invalid argument' error. The referenced node's code didn't have a working example involving sysread.

Here's my test code, using a local temporary file instead of the block device: 
#!/usr/bin/perl

use strict;
use warnings;

$|++;

use Fcntl qw(:DEFAULT O_ASYNC O_DIRECT);
use Sys::Mmap;

my $FH;
my $msg;
my $BUFFER="";
my $ret;
my $seekpos=2;
my $BUFSIZE = 4096 ;
my $soffset;

sysopen($FH, "/tmp/test.dat", O_RDWR | O_ASYNC | O_DIRECT | O_CREAT, 0
+666) or die "$!";
substr($BUFFER, 0, $BUFSIZE) = "\0"x$BUFSIZE;

$soffset = $BUFSIZE * $seekpos;
print "seeking to $soffset\n";
$ret = sysseek($FH, $soffset, 0) or print "first sysseek FAILED: $!\n"
+;
print "first sysseek succeeded.  returned $ret\n";
$msg="data at $soffset";
substr($BUFFER, 0, length($msg))=$msg;
$ret = syswrite($FH, $BUFFER, $BUFSIZE) or print "first syswrite FAILE
+D: $!\n";
print "first syswrite succeeded.  returned $ret\n" if defined $ret;

print "\n";
print "mmap coming up...\n";
$ret=mmap($BUFFER, $BUFSIZE, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANO
+N, STDOUT) or die "Couldn't mmap: $!\n";
printf "mmap succeeded.  returned address %x\n", $ret if defined $ret;
if (($ret % 4096) == 0) {
        print "coolness... $ret is aligned to 4096 byte boundary\n";
}

print "\n";
print "seeking to $soffset\n";
$ret = sysseek($FH, $soffset, 0) or print "second sysseek FAILED: $!\n
+";
print "second sysseek succeeded.  returned $ret\n" if defined $ret;
$msg="data at $soffset";
substr($BUFFER, 0, length($msg))=$msg;
$ret = syswrite($FH, $BUFFER, $BUFSIZE) or print "second syswrite FAIL
+ED: $!\n";
print "second syswrite succeeded.  returned $ret\n" if defined $ret;

print "\n";
print "seeking to $soffset\n";
$ret = sysseek($FH, $soffset, 0) or print "third sysseek FAILED: $!\n"
+;
print "third sysseek succeeded.  returned $ret\n" if defined $ret;
$ret = sysread($FH, $BUFFER, $BUFSIZE) or print "sysread FAILED: $!\n"
+;
print "sysread succeeded.  returned $ret\n" if defined $ret;

exit 0;

[download]

As you can see if you run the test, Sometimes the first syswrite fails, which is expected because the buffer hasn't been aligned yet with mmap. The second syswrite always succeeds, but the sysread on the same filehandle and using the same aligned buffer _always_ fails. What can I do to effectively use sysread on an O_DIRECT filehandle?

Erik Schorr
sub email { scalar reverse "gro.apra\@erik" }

In reply to sysread/syswrite and O_DIRECT alignment problem by fishnuts

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.