in reply to Does perl have a builtin limit to the size of shared memory segments I can write to?

Yes, this looks like a bug in Perl.
Perl_do_shmio(pTHX_ I32 optype, SV **mark, SV **sp) { #ifdef HAS_SHM char *shm; struct shmid_ds shmds; const I32 id = SvIVx(*++mark); SV * const mstr = *++mark; const I32 mpos = SvIVx(*++mark); const I32 msize = SvIVx(*++mark);

The parameters are stored into variables declared as I32, rather than SSize_t.

The mem-read and mem-write are performed in Perl rather than a system call, so there's no reason to truncate the offset and length to 32-bit.

if (optype == OP_SHMREAD) { char *mbuf; /* suppress warning when reading into undef var (tchrist 3/Mar/00) + */ SvGETMAGIC(mstr); SvUPGRADE(mstr, SVt_PV); if (! SvOK(mstr)) SvPVCLEAR(mstr); SvPOK_only(mstr); mbuf = SvGROW(mstr, (STRLEN)msize+1); Copy(shm + mpos, mbuf, msize, char);

I'm afraid you won't be able to get a fix for this unless you recompile your own perl, but please consider filing a bug report so that perl 5.42 could have it fixed.

Meanwhile, there is File::Map which is basically the same thing, except you have to create your own files (in tmpfs if you want them to be pure RAM). You also have to be careful not to accidentally copy that mapped scalar, which would be expensive.

  • Comment on Re: Does perl have a builtin limit to the size of shared memory segments I can write to?
  • Select or Download Code

Replies are listed 'Best First'.
Re^2: Does perl have a builtin limit to the size of shared memory segments I can write to?
by Anonymous Monk on Jan 08, 2025 at 09:57 UTC

    Thanks for the prompt reply.

    I opened an issue about that.

    Recompiling my own perl is not an option, as I aim to share the code and attain some degree of portability (on Linux, no plans to extend this to other OS). I will carefully look into the File::Map option, or perhaps, I can find a simpler workaround by splitting my large index over a few shared memory segments, for now.

    Best,

    Valerio

      Thank you very much for opening it!

      Just for reference, the issue is GH #22895.