in reply to SysV shared memory (Look-Alike) -- pure perl

There's hardly any reason to use SysV shared memory these days. Not for performance, anyway. Plain mmap of a regular file, or anonymous mapping (parent/child scenario) will accomplish much the same. The SysV interface could be convenient in some cases; one example is/was postgres:

PostgreSQL uses System V shared memory, because it provides a feature that is available via neither of the other two systems: the ability to atomically determine the number of processes attached to the shared memory segment.

In your code, the shmwrite() usage is not particularity efficient. (On Linux) each of those writes translate to three system calls — shmctl/shmat/shmdt — and a full memcpy/memset of the segment. IPC::SysV provides shmat, memwrite; the idea of shared memory is to avoid syscalls in the first place.

Finally, to benchmark message passing, one needs both a reader and a writer...

Replies are listed 'Best First'.
Re^2: SysV shared memory (Look-Alike) -- pure perl
by flexvault (Monsignor) on Jul 23, 2014 at 15:28 UTC

    oiskuu,

    Thanks for the response! Some points:

      There's hardly any reason to use SysV shared memory these days.
    Hopefully, that was my point in the first place. I wanted to have an alternative that didn't require SysV ( or mmap ) operating system support. My 1st attempt didn't work since I used both buffered and unbuffered I/O.

      PostgreSQL uses System V shared memory, because it...

    That is how I'm hoping to use this technique. I have a pure-Perl database engine that is basically single user for writing and multi-user for reading within the same environment. It's very fast, but when any single DB within an environment goes past 10MM records the writes drop sharply. Up to about 10MM records the writes are about 3,000/second. Above 10MM records that drops to 1,000 writes/second or worse. But that's not the problem either. The problem is that when the 'writer' is taken longer, the multiple 'readers' performance degradates significantly. Reading sequentially ( 'ReadNext' or 'ReadPrev" ) within the same DB the 'readers' usually perform about 100K reads / second. Above 10MM the reads drop to 20-25K reads / second. This technique would be to allow the writer to indicate which tree leaf the writer is working within. If the reader needs a record from that leaf, they have to wait, but if not then they can proceed with any other leaf.

      to benchmark message passing, one needs both a reader and a writer...

    Agreed, but I was testing that the writers didn't clobber the shared memory. In production, I can 'flock' with 'LOCK_SH' for readers. Each tree leaf is independent of any other tree leaf, so maybe I'll be able to have multi-writers

    I'm going to add the code in the main thread, but I'd appreciate any comments on what it is doing.

    Regards...Ed

    "Well done is better than well said." - Benjamin Franklin

[reply]

In Section Cool Uses for Perl