Is that 100 requests/sec sustained or burst?

I have to agree with Corion's design here.

collector (mod_perl) -> MySQL -> sender (LWP?)

Use a mod_perl handler with persistent database connections (ala Apache::DBI) to write the incoming info to a suitably fast database (like MySQL). Have a separate daemon written in Perl (or something compiled if performance dictates) that reads from the DB and sends the responses.

Make sure the DB stores timestamps for request received and response transmitted. Your sending daemon fetches a 'worklist' by querying for records w/o transmitted timestamp. The table can be archived/summarized/cubed periodically during slow periods. If there are no slow periods, you might need to investigate 'breaking-the-mirror' type solutions to get an offline snapshot for reporting.

--Solo

--
You said you wanted to be around when I made a mistake; well, this could be it, sweetheart.

In reply to Re^2: Apache or Daemon in Perl ? by Solo
in thread Apache or Daemon in Perl ? by szabgab

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.