Occasionally I want to work with a big data set, too big for even the humongous memory on my late-model COTS supercomputer.

Now is such a time, and I am considering writing an Inline::C module that will mmap my massive fixed-length-record data set and provide an @-overloaded or otherwise magical object that will look like an array, to perl, but will pull fixed-length records out of my file. Surely this has been done already; use of Sys::Mmap and substr would take me pretty much there, something like

use Sys::Mmap;
use constant BDfname=>"bigdatafile";
use constant BD_RECORDSIZE=>9876;
-s BDfname % BD_RECORDSIZE and die"size ne mult of recsize";
open BIGDATA, BDfname or die $!;
mmap(
    $BD,
    -s BDfname,
    PROT_READ,
    MAP_SHARED,
    BIGDATA
);
sub BDrec($){
   substr($BD,BD_RECORDSIZE*$_[0],BD_RECORDSIZE)
}
[download]

might work. But still,

My question is, what would happen if perl_mymalloc was replaced with something that created and memory-mapped files instead of doing mallocs?

Would I get a fully checkpointable perl, able to gracefully deal with larger-than-memory data sets without eating up all my swap (of course it might still thrash, but not on the designated swap space)?

How severe would be the speed hit? (before hitting the memory limit)

Has this been done already?