Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re: Shared Memory Cache or Database?

by Corion (Patriarch)
on Dec 07, 2020 at 16:15 UTC ( #11124781=note: print w/replies, xml ) Need Help??

in reply to Shared Memory Cache or Database?

Assuming that you plan to run all of this on a single machine, another interesting approach could be to just use shared memory among all processes and to do away with using Perl memory structures and instead use a fixed memory layout.

This makes some assumptions about the number of assets, markets and brokers, but basically imagine a (large) string indexed by fixed offsets where you write the (packed) prices to.

The downside is obviously the inflexibility between the server and clients, because you need to update them at once whenever a new market, asset or broker comes along.

If you use shared memory, you will get no backlog as it goes away when the last process is stopped. This may be an up- or downside.

Replies are listed 'Best First'.
Re^2: Shared Memory Cache or Database?
by zapoi (Novice) on Dec 07, 2020 at 16:40 UTC
    Hi Corion,

    Indeed the processes will be running on one machine.

    I considered that approach but I'm not sure that I could efficiently code it in this way, as there will be varying number of brokers depending on the market, and some might not be publishing prices all the time, it also will make reusing the memory up once a market falls out of interest challenging. I can see myself getting in all kinds of hashes of index values to try and keep track of where the current and previous data snaps are stored.

    Please let me know if I've completely misunderstood and my concerns are ill founded.


      Yes, if brokers are somewhat dynamic, and you can't use the same set of brokers for all markets (etc.), calculating the index where to store a price becomes quite cumbersome/error-prone.

      Reusing the memory allocated to a market could be done on a time slice, but all of that depends on how dynamic the data really is.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11124781]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2022-05-25 12:24 GMT
Find Nodes?
    Voting Booth?
    Do you prefer to work remotely?

    Results (90 votes). Check out past polls.