Hi,

I'm designing a algorithmic trading app and would appreciate any wisdom that you would be so kind as to bless upon me.

I started with the following design:

main program, using IPC::Shareable create and tie %cache.

Run a sub dataCollector in a new fork, which loops every 0.1s incrementing $i, and then adds the data to a hash giving a {$marketId}{$i}{$broker}{price}. Each time a new set of data is published send a SIGUSR1 to the other processors to alert them that they have new work to do. Once a market is out of it's window of interest approximately 30 minutes later delete $cache{marketId}. Each 0.1s loop yields about 6k of data, and there may be upto 8 markets that are of interest simultaneously to total memory requirement is about 420Mb.

Main program will call subs for each strategy, which will fork and return the forked pid to to main. The strategy workers will receive their interrupts and handle them by reading the shared memory, do various computations to decide if the conditions are good to trade or not.

The dataCollector is the only process that would need to write to the cache, and each worker would be triggered simultaneously and be reading the same piece of memory. Another worker task will be writing the data out to file for back testing strategies, however if data is lost it's not an issue.

Hope that makes sense in what my objective is. What I would appreciate your wisdom on is:

1. IPC::Shareable doesn't seem to handle deeply nested hash refs very well and probably therefore isn't the right choice. Is there an alternative that I should consider?

2. Most of my reading has pointed me towards using an in-memory database. In your experience which free database would give me the fasted throughput in this scenario?

Please turn your flame throwers up to max and burn this down, and help me get to the right foundations.

Thanks

Zapoi

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.