Re: mmap in Perl?

Thanx for above Re^4, RobinV. This contained the missing info & confirmed the missing reader to be my suspected kitten, ahem - cat :). Btw. did you notice that your indenting didn't quite survive pasting?

Observation: It looks to me like $mmap isn't synced to file (or maybe even to the mmapped memory?) if the length changed, even if the new value _remains_ usable within the script. Modify your writer similar to the scrap below to make it work, but choose something more sensible than my substring mess.

# WORKS -- but it's suddenly quite slow due to synchronous sync :)
for(my $i=0;$i<105;++$i) { 
   substr($mmap,0,2) = substr($i . (" "x20) ,0,2); 
   sync($mmap,1); 
}
[download]

Question: isn't this a bug / incomplete documentation? IMHO there should be a way to at least optionally turn on a warning or error somehow for this case, otherwise that's a good place for hard-to-find bugs. (I also forwarded the URL for this thread to the module's author for a rain check. Maybe I missed something when skimming the module's documentation)

Maybe worth a check: There are more mmap modules on CPAN besides File::Map (Cache::FastMmap, Sys::Mmap + ::Simple, IPC::Mmap) some of which may turn out to be less rigid in assignments, maybe allowing implied remapping, padding or substring-insertion for shorter assignments.

Comment on Re: mmap in Perl? Select or Download Code

Replies are listed 'Best First'.
Re^2: mmap in Perl? by Leon Timmermans (Acolyte) on Oct 21, 2009 at 20:21 UTC
I´m the author of File::Map. Due to circumstances I can't test or fix this right now, that will have to wait until next week. This definitely is a bug, I´ll fix it as soon as I´m back home. Data loss as is observed here is not acceptable. Something seems to go wrong in the detection of direct assignment, it should have given a warning and then fix things up. As for your second question: all other memory mapping modules either have the same issue of requiring substr() assignment to work properly, or use ties which cause the file to be copied completely on each read or write. The latter clearly isn't any better than the former IMSNHO.	[reply]
Re^3: mmap in Perl? by jakobi (Pilgrim) on Oct 21, 2009 at 20:57 UTC
Thanx for checking in, Leon. It's rather as if something detects the assignment of wrong size and silently refuses. So from outside, bug #1 is either in refusing EXOR in being silent about refusing. What kind of behaviour do you intend to provide on wrong-sized assignments (too large, too short, undef <=> replace & pad, overwrite beginning, resize mapping)? I don't remember anything in the module docs about what to expect, IIRC (buglet #2, either mine or your's :) ). cu & thanx, Peter	[reply]
Re^4: mmap in Perl? by Anonymous Monk on Oct 22, 2009 at 12:05 UTC
Bug 1: Due to the way perl's internals work, it is impossible to check the new value until after the assignment has happened. If the assignments goes wrong, perl itself reassigns the variable to another piece of memory. The way I deal with that is checking after every assignment. For some unknown reason that doesn't happen in this case. In other words, the problem is that it doesn't detect what's going on. Bug 2: It is supposed to give you a warning when you're doing something like that. I considered making it an exception, but that may be a little	[reply]
Re^2: mmap in Perl? by RobinV (Sexton) on Oct 21, 2009 at 07:29 UTC
Thanks for the reply Btw. did you notice that your indenting didn't quite survive pasting? No I didnt.. I checked.. and well I do use `sync($mmap,1);` Maybe worth a check: There are more mmap modules on CPAN besides File::Map Well I had a look about, I found one thats good. However coming from a C world I thought of mmap ;) and D-Bus but DBus doesnt have documentation in perl. My code works! ^^ Thanks	[reply] [d/l]
Re^3: mmap in Perl? by jakobi (Pilgrim) on Oct 21, 2009 at 09:25 UTC
@Robin: The "indenting not surviving paste" refers to the leading whitespace in this code scrap `> 010: map_file $mmap, $file, '+<'; > 011: if (!defined($mmap)) { print "ERROR!!\n"; } > 012: for(my $i=0;$i<10000;++$i) { $mmap = $i; sync($mmap,1); + } > 013: unmap ($mmap);` [download] all the statements are at the same level, but your indent above implied differently `map_file $mmap, $file, '+<'; if (!defined($mmap)) { print "ERROR!!\n";} for(my $i=0;$i<10000;++$i) { $mmap = $i; sync($mmap,1); } unmap ($mmap);` [download] cu, Peter (wondering how he mismanaged his browser to get a duplicate node, but then using it to add the note on indenting here instead of asking for a reap).	[reply] [d/l] [select]
Re^3: mmap in Perl? by jakobi (Pilgrim) on Oct 21, 2009 at 09:29 UTC
DBUS? Then your question was more a XY question :). While I can see shared memory to DBUS, my take of mmap is more about efficiency and accessing files other than with print&sysread. You might wish to post a separate question about DBUS. CPAN offers at least one module for this: Net::DBus. What I can offer on DBUS is a quick hack I did a while ago: check if I can see & send dbus events and somehow use them with waitcond - a rather primitive dbus example. Note that all I needed was the ability to grep and tail dbus, and thus a bit of glue, shell and an existing dbus monitoring tool was enough to conclude that experiment & obtain a useful but missing DBUS debug tool.	[reply]