http://qs1969.pair.com?node_id=11148131

xiaoyafeng has asked for the wisdom of the Perl Monks concerning the following question:

Hi Esteem Monks!

If I know correctly, perl has many type of variable: lexical variable, package variable, interpreter variable. but it lacks a real global variable(crossing multi-interpreter)

no real gloabl variable on perl maybe not a big problem, but my question is, why perl doesn't have a management container on perl interpreter? If perl have a management container, we can do more (I guess) to manipulate interpreter: share memory, attach/detach interpreter, resource management(GC?) etc. A container may slightly affect the performance, but it can let perl having more power and better face on the modern multi-core environment. At least maybe we don't need see the discourage label on threads for next 5 years?

it seems Coro::Multicore does this work more or less, but it only can be used on XS.

Is there some difficulty on this? or maybe this feature wouldn't help perl?, Whatever the reason, Please enlighten me out. Thanks!





I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

  • Comment on having a container on perl VMs is good ?

Replies are listed 'Best First'.
Re: having a container on perl VMs is good ?
by kcott (Archbishop) on Nov 11, 2022 at 11:34 UTC

    G'day xiaoyafeng,

    Unfortunately, this seems to fall into "XY Problem" territory: you've indicated that you think "management container" is a solution (Y); you haven't specified what problem (X) you want this to solve (except in a very general sense).

    "... let perl having more power and better face on the modern multi-core environment."

    That sounds like what MCE achieves. This has a lot of documentation: it's nicely laid out, and has many examples (including a specific MCE::Examples page), but will still require a fair bit of reading. A couple of suggestions that seem to fit with what you're asking: "GLOBALLY SCOPED VARIABLES AND MCE MODELS"; "MCE->gather()".

    "... container on perl VMs ..." [from title]

    I was unsure what you meant by this. For $work, I've set up a VM in which I run applications that use Docker images with 15 different Perl versions. Each image uses code & data specific to its own Perl version but can also access more general, shared code & data from the VM. Perhaps this is closer to what you want.

    There's also "perlipc - Perl interprocess communication". That may provide some suggestions that might be of use to you.

    — Ken

      Another option that i nearly always use* is Net::Clacks. It allows asyncronous message broadcasting between processes. It also supports storing values on the Net::Clacks Server, similar to memcached. It supports network and Unix domain sockets (for better speed).

      * I developed that thing. Yeah shameless self promotion here, sue me...

      PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP

        Yes, although it would've been a while ago now, I do remember you posting that. The Discworld: Going Postal reference was not lost on me. :-)

        — Ken

Re: having a container on perl VMs is good ?
by Corion (Patriarch) on Nov 11, 2022 at 09:42 UTC

    Are you looking for threads?

    If you want to have a central thing for keeping values across interpreters, look at a database, memcached or Redis maybe?

    I'm not sure what problem you are trying to solve.

      basically, what I mean is multi interpreters is running on one process.

      If you want to have a central thing for keeping values across interpreters, look at a database, memcached or Redis maybe? I'm not sure what problem you are trying to solve.

      multi interpreters on one process have many advantages: 1. all the way you point out has data copy, 2. real threads on windows but not fork-like one.




      I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

        If you want multiple interpreters running in one process, that is what threads gives you. See also threads::shared, and also Thread::Csp for a more shared-nothing approach.

Re: having a container on perl VMs is good ?
by NERDVANA (Deacon) on Nov 12, 2022 at 00:43 UTC
    It is true that you can only get cross-thread global variables using XS because Perl does not have a synchronization API to make sure only one thread modifies data at a time. C/++ does have this feature, so you can add it to Perl using XS modules.

    (Edit: I stand corrected, Perl does! threads::shared is a core perl module)

    Now, you say it would give perl more power for multi-processing if Perl did have this, but you need to make a stronger argument to convince us. To start with, many of us do not even use perl threads in the first place, and may even disable the feature in our perl interpreter to make it run faster. This is not because we aren't familiar with the idea (I did a lot of Java back in college) but because we have found other ways to solve our problems using perl.

    For shared structured data, most of us use a database. There are many databases to choose from, too. You can of course run Postgres and create a table with a 'json' column, and write arbitrary data to it, or you can use a database-file like LMDB_File or SDBM, SQLite etc. In these cases perl memory-maps the file data and serializes and deserializes on each read/write. If you need locking, you can implement it with flock.

    For shared flat data, you can just File::Map it into each process or thread, and have high-speed un-synchronized access. This is most useful with large static data that doesn't need synchronized anyway.

    For the few types of problems where people really do want this, they probably also want more speed than plain perl. The XS modules give you a lot of speed in addition to extra threading features.

    While it used to be common for people to implement high-speed transactional behavior inside of a single process, I just haven't needed to do that in the last 15 years that I've been using perl. Almost every modern application needs the potential to scale up beyond one process on one host, so I pretty much start every project assuming I will use some type of database.

    Maybe you can describe the problem you're trying to solve that would benefit from tight single-process relay of data structures between threads?

      Not to detract from your points, threads::shared has synchronization primitives and also ensures that a shared variable is written to by only one thread at a time.

      But IMO when you start using / needing these, something went wrong in the overall design already. I prefer to structure my thread code around Thread::Queue, which changes the design to something message-based and which is far easier to reason about than variables that get updated by multiple threads.