zentara has asked for the wisdom of the Perl Monks concerning the following question:

Hi, In a thread today, I saw mention of Sys::Mmap, and after reading it's perldoc, which says:

The Mmap module uses the POSIX mmap call to map in a file as a Perl variable. Memory access by mmap may be shared between threads or forked processes, and may be a disc file that has been mapped into memory. Note that PerlIO now defines a ":mmap" tag and presents mmap'd files as regular files....

This sounds very interesting for sharing data, but I can't seem to figure out how to put it to good use. Does anyone have any simple, clear examples of it's use, or is it best left in obscurity?

I'm not really a human, but I play one on earth. flash japh

Replies are listed 'Best First'.
Re: anyone have some good mmap examples?
by perrin (Chancellor) on Dec 12, 2004 at 19:03 UTC
    You can probably find an example in Cache::Mmap, but if you just want to share data (rather than learn about mmap), use Cache::FastMmap, which is written mostly in C and is very fast.
[reply]
Re: anyone have some good mmap examples?
by sgifford (Prior) on Dec 13, 2004 at 06:42 UTC
    I often use mmap for very fast file I/O. Here's an example of a simplistic grep, which is not quite twice as fast as a simple perl -ne 'BEGIN { $pat = shift } print if /$pat/o' on my system. 
    #!/usr/bin/perl

use warnings;
use strict;

use Sys::Mmap;

our $pat = shift;

foreach my $a (@ARGV)
{
  my $mmap;
  if (! -f $a)
  {
    warn "'$a' is not a regular file, skipping\n";
    next;
  }
  open(F, "< $a")
    or die "Couldn't open '$a': $!\n";
  mmap($mmap, 0, PROT_READ, MAP_SHARED, F)
    or die "mmap error: $!\n";
  while ($mmap =~ m/($pat)/omg)
  {
    my $pos = pos($mmap)-length($1);
    # Find the beginning and end of this line
    my $first = rindex($mmap,"\n",$pos)+1;
    my $last  = index($mmap,"\n",$pos)+1;
    print substr($mmap,$first,$last-$first);
  }
  munmap($mmap)
    or die "mmunmap error: $!\n";
  close(F)
    or die "Couldn't close '$a': $!\n";
}

    [download]

    That doesn't really address sharing, though. Basically, if two programs mmap in the same file, when one makes changes the other will be able to see them, which provides a simple shared memory region.

    Update: Fix bug matching first line of file.

[reply]
[d/l]
[select]
Re: anyone have some good mmap examples?
by kscaldef (Pilgrim) on Dec 15, 2004 at 02:58 UTC
    The simplest example would simply be a large read-only data file that needs to be accessed by multiple processes. For speed, you want each process to have it in memory rather than hitting disk all the time. However, you don't want to waste memory giving each process a copy. So, you mmap it and everyone can see it, and have fast access, but you only use the memory once.
[reply]
      Now, why couldn't they have just said that in the docs? :-)
      I'm not really a human, but I play one on earth. flash japh
[reply]
        Because there's lots of more complicated uses too. For example, maybe the data isn't read-only. Then you need to consider how to handle writes. Or it might not be a simple data file. For example, I know an application that uses finite automaton data structures that weigh in a hundreds of MB. The whole data structure is dumped to disk for storage between runs. When the data structure is loaded into the applications it uses an mmapped region.
[reply]

Back to Seekers of Perl Wisdom