I think you are taking the wrong approach. If you do thing right then most of the heavy lifting should take place inside your relational database engine (MySQL in this case).

As you said, there is a large performance penalty when DBIx::Class creates objects from rows in the database. This is a known bottleneck with any Object relational mapper. It exists because by choosing to use DBIC, you have traded a little performance for convenience. It is hitting you hard because you have created lots of objects.

What I think you should be doing is designing your database schema, and writing better queries so that DBIC only inflates the objects you really need. For example could you pre-calculate some stuff and store it as extra rows in your database so that calculations happen faster? Could you indexes or views to your database to further speed things up?

Once you have succeeded in shifting the load to MySQL, then your problem will become one of optimising the performance of that database, which is not something we are expert in, though I suspect that by the time you reach that stage, you will have gained acceptable performance as you will have eliminated the DBIC object inflation penalty.

In reply to Re: Hardware to run heavy dbix::class stuff? by chrestomanci
in thread Hardware to run heavy dbix::class stuff? by uG

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.