in reply to reading from a huge file

You can always use Tie::File and tie to the file. (See especially mode for finer control.)

Something along the lines of (taken pretty much straight from the POD):

use Tie::File; use Fcntl 'O_RDWR'; tie my @contents, 'Tie::File', $ARGV[0], mode => O_RDWR; # Assumes you pass the file name on the command line

You can now cycle through @contents like a typical array, each field being a line of your file. Be careful, the mode passed will determine the sort of changes (if any) you can make to the file, whether intentional or otherwise.

Both Tie::File and Fcntl are core modules, so no CPAN necessary.

While Fcntl isn't entirely necessary, it will help you maintain finer control over how the file is accessed on disk. Again, see the 'mode' section of the documentation.

Notes: strict and warnings implied. I haven't tried this with a file of the size you are dealing with, but have had success with other large files (>4G) in the past.

Replies are listed 'Best First'.
Re^2: reading from a huge file
by BrowserUk (Patriarch) on Mar 21, 2011 at 20:40 UTC

    Please do not suggest the use of Tie::File for use with files bigger than a few tens of megabytes.

    Using it makes processing such files 10s or 100s of times slower than using normal line-by-line access for no benefit.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Very good to know, BrowserUK. Thanks for the clarification!