(tye)Re: 4Gb filesize limit

See SetFilePointer() in Win32API::File (which comes standard with Win32 Perl).

Note that using Perl file handles with this may present a problem. You might be able to simply seek before each call to SetFilePointer():

use Win32API::File qw( SetFilePointer GetOsFHandle );
sub BigTell {
    my( $fh )= @_;
    seek( $fh, 0, 1 );    # Flush buffers
    my $osf= GetOsFHandle( $fh );
    my $hi= 0;
    my $lo= SetFilePointer( $osf, 0, $hi, 1 );
    return pack "NN", $hi, $lo;
}
sub BigSeek {
    my( $fh, $pos, $whence )= @_;
    my $osf= GetOsFHandle( $fh );
    seek($fh,0,1);    # Flush buffers
    my( $posHi, $posLo )= unpack "NN", $pos;
    $posLo= SetFilePointer( $osf, $posLo, $posHi, $whence )
      or  return;
    return pack "NN", $posHi, $posLo;
}
[download]

Note that I chose a format for the "big pointer" such that string comparisons are meaningful.

Unfortunately I don't have time at the moment to test this. However, if you have problems with it, reply and I'll likely be able to help.

On the other hand, please reply if this turns out to work well as it'd make a good addition to Win32API::File. (:

Updated to fix a typo.

- tye (but my friends call me "Tye")

Comment on (tye)Re: 4Gb filesize limit Download Code

Replies are listed 'Best First'.
Re: (tye)Re: 4Gb filesize limit by ruffing (Acolyte) on Dec 06, 2001 at 20:42 UTC
I'm still confused, what's the purpose of these two functions? BigTell seems to move the file pointer to the beginning and return 00000000 while BigSeek seeks a file pointer to a specific location based on the $pos you pass in, which I try to use something along the lines of "00010000" for the beginning of the next 4Gb segment. What read call do I use from here to actually read in that segment?	[reply]
(tye)Re2: 4Gb filesize limit by tye (Sage) on Dec 07, 2001 at 00:22 UTC
BigTell seems to move the file pointer to the beginning No, in BigTell(), the last arguments both to seek and to SetFilePointer() is 1, not 0, so those calls both seek 0 bytes from the current position; that is, neither call should reset the file position. The call to seek is there for the side effect of flushing read/write buffers and the call to SetFilePointer() is there for the side effect of obtaining the current position, even if it is past the 4GB mark. In other words, BigTell() should act just like tell except that it works far past the 4GB mark and so returns the current position as an 8-byte string instead of as an integer. And thanks for replying. It got me to test the code and find a typo in BigSeek() [I forgot to rename one instance of $posHigh to $posHi] and to update the original code to fix that. I try to use something along the lines of "00010000" for the beginning of the next 4Gb segment. Actually it would be `pack( "NN", 1, 0 )` which is "\0\0\0\01\0\0\0\0" for the start of the 4GB segment. What read call do I use from here to actually read in that segment? I was thinking you could just use normal Perl I/O operations. But now that you mention it I guess I do recall hearing that those don't work at least some of the time. I'd think that at least sysread would work, but I've certainly been wrong before and it would take me a while to generate a 5GB file in order to test that. It makes some sense that buffered I/O operations could fail, but I was surprised to hear that just reading a huge file like a stream (even with buffering) fails. Though you could certainly use ReadFile() [and WriteFile()] from Win32API::File. Note that if you do need to use those, then you might also need to comment out the calls to seek in both BigSeek() and BigTell() as those might also break in such cases. - tye (but my friends call me "Tye")	[reply] [d/l]