perlofwisdom has asked for the wisdom of the Perl Monks concerning the following question:

I have an HP-UX box (B.11.23) with perl v5.8.8. perl -V:uselargefiles reveals that large file support is enabled (uselargefiles='define';). The trouble is that file tests on files > 2G don't work (ie, "if (-e $filename) {...}"). The test returns "Value too large to be stored in data type" in the $! variable. Any ideas? Thanks.
  • Comment on File test (-e) fails on files > 2G, uselargefiles=define

Replies are listed 'Best First'.
Re: File test (-e) fails on files > 2G, uselargefiles=define
by almut (Canon) on Aug 07, 2007 at 00:09 UTC

    Could you show the full output of "perl -V"?  Unfortunately, there are a number of subtleties that can go wrong with large file support... (see this document for some background info — in particular chapter 6). In other words, it's hard to tell what the problem might be without having all the details... (and even then, it's not always easy :)   BTW, did you build that perl yourself (if not, where did you get it from)?

      I did not build it myself. One of our SAs must have installed it, but I don't know from what source. Here is the complete output from perl -V...
      Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=hpux, osvers=11.23, archname=PA-RISC2.0 uname='hp-ux waves b.11.23 u 9000800 2510705975 unlimited-user lic +ense ' config_args='-A prepend:libswanted=cl pthread -d -e -Dcc=gcc -Dpr +efix=/usr/local useposix=true' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultipl +icity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_HPUX_SOURCE -mpa-risc-2-0 -fno-strict-alias +ing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGE +FILE_SOURCE', optimize='-O0', cppflags='-D_HPUX_SOURCE -D_HPUX_SOURCE -mpa-risc-2-0 -fno-strict- +aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include' ccversion='', gccversion='4.1.1', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1 +6 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', + lseeksize=4 alignbytes=8, prototype=define Linker and Libraries: ld='/usr/bin/ld', ldflags =' -L/usr/local/lib -L/opt/local/lib' libpth=/usr/local/lib /opt/local/lib /lib /usr/lib /usr/ccs/lib libs=-lcl -lpthread -lnsl -lnm -lndbm -lgdbm -ldb -lmalloc -ldld - +lm perllibs=-lcl -lpthread -lnsl -lnm -lmalloc -ldld -lm libc=/lib/libc.sl, so=sl, useshrplib=false, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_hpux.xs, dlext=sl, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl +,-B,deferred ' cccdlflags='-fPIC', lddlflags='-b -L/usr/local/lib -L/opt/local/li +b' Characteristics of this binary (from libperl): Compile-time options: PERL_MALLOC_WRAP USE_LARGE_FILES USE_PERLIO Built under hpux Compiled at Sep 19 2006 17:04:15 @INC: /usr/local/lib/perl5/5.8.8/PA-RISC2.0 /usr/local/lib/perl5/5.8.8 /usr/local/lib/perl5/site_perl/5.8.8/PA-RISC2.0 /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl .

        I suspect you should also have -D_FILE_OFFSET_BITS=64 in addition to -D_LARGEFILE_SOURCE ...

        Maybe the problem is that this version had been built in a 32-bit build environment (gcc version), at least the README.hpux that comes with the Perl sources states that (emphasis added)

        Using Large Files with Perl on HP-UX

        ... Three separate methods of doing this are available. Of these methods, the best method for Perl is to compile using the -Duselargefiles flag to Configure. This causes Perl to be compiled using structures and functions in which these are 64 bits wide, rather than 32 bits wide. (Note that this will only work with HP's ANSI C compiler. If you want to compile Perl using gcc, you will have to get a version of the compiler that supports 64-bit operations. See above for where to find it.)

        Anyhow, before you go to the trouble of building your own version, you might try one the binaries generously offered for download by our very own cbu.

        According to the uname signature "hp-ux waves b.11.23 u 9000800 2510705975 ..." the perl you have seems to come from the HP archive... (so there's probably no point in giving that one another try).

Re: File test (-e) fails on files > 2G, uselargefiles=define
by Corion (Patriarch) on Aug 08, 2007 at 09:40 UTC

    As a possible (ugly) workaround, you can resort to the shell or other tools to perform the tests and file reading you need done:

    perl -le 'print system(qw( test -e fileThatDoesNotExist ))' 256 perl -le 'print system(qw( test -e .profile ))' 0

    ... and for reading from a file (seek()ing is not supported) :

    open my $fh, qq(cat "$file" |) or die "Couldn't read '$file': $!";

    Of course, you will like to move to a proper perl as soon as possible, but this might be a stopgap measure.