in reply to Re^2: reading from a huge file
in thread reading from a huge file

I think the OP's basically stuffed ...

This is FUD. Didn't you notice I said (>4GB)?

Every version of Perl I've used on windows in the last 9+years has been built with USE_LARGE_FILES.

Here is perl processing a 12GB/141 million line file using Active State 5.10.1 in 88 seconds: 

C:\test>dir dna.txt
 Volume in drive C has no label.
 Volume Serial Number is 8C78-4B42

 Directory of C:\test

08/11/2010  16:35    12,831,000,000 dna.txt
               1 File(s) 12,831,000,000 bytes
               0 Dir(s)  281,938,862,080 bytes free

C:\test>perl -nE"}{say $." dna.txt
141000000

[download]
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^4: reading from a huge file
by educated_foo (Vicar) on Mar 21, 2011 at 19:51 UTC
    So you've always had Perls compiled with USE_LARGE_FILES. How is that relevant to the OP? And how is what I wrote "fear, uncertainty, and doubt?"
[reply]
      How is that relevant to the OP?

      Because, by default, every windows perl build is configured with large file support. To build without it require manual intervention, and there is no good reason for doing so. And the vast majority of Perl installations on Windows are AS binary distributions that all come with large file support.

      So, whilst it isn't impossible for the OP to be using a perl build that has been deliberately crippled, the likelyhood is small.

      This:

      I think the OP's basically stuffed unless he can get another program to chop the file into smaller pieces,

      Is FUD because:

      • because it demonstrates a complete lack of understanding of Perl on Windows.
      • it assumes just about the most unlikely reason for the problem based upon no more information than the mere mention of "huge file".
      • even if your assumption as to the cause of the problem turns out to be a lucky correct guess, you offer the very worst solution to that problem.
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        Because, by default, every windows perl build is configured with large file support.
        If only you were right... Tragically, anything older than 5.8.4 compiled with Borland can't use large files at all, whatever it does "by default."

        Anyways, let's see what the OP's actual problem is before allocating the egg to someone's face.

[reply]

In Section Seekers of Perl Wisdom