in reply to Re: mmaping a large file
in thread mmaping a large file

« If you have a large dataset in a file: and a) only need access to small bits of it in any given run; b) you can find those bits without reading through the whole file from the beginning; then mapping can be an effective way on minimising the number of pages read from disk. »

yes, that's exactly the use case.

I've just tried:

use Sys::Mmap;
new Sys::Mmap my $f, 8192, q(bigfile);

[download]

but now I get an error during cleaning:

(in cleanup) munmap failed! errno 22 Invalid argument

This is not going to be easy, is it?

Replies are listed 'Best First'.
Re^3: mmaping a large file
by BrowserUk (Patriarch) on Aug 24, 2012 at 00:26 UTC

    Warning: I don't use *nix, so I cannot test anything I'm about to say.

    The first thing I notice is that the POD for Sys::Mmap uses 

    new Mmap my $f, 8192, q(bigfile);

    [download]

    not

    new Sys::Mmap my $f, 8192, q(bigfile);

    [download]

    I would have expected you to receive a compile-time error message from that, which suggests you aren't using strict/warnings. It might be a good idea to start.

    The other thought is that mapping a 2GB file through a 8k window is going to involve a lot of shuffling.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom