Hello Monks-

I have 2 perl applications running on different hosts which need to send information (perl hashes) from one to another. What is the best way to do this?

Here's what I have implemented as a prototype:

Application A takes a constant stream of log data, cooks it (into perl hashes) and writes them to an ascii file in JSON format. This is done in one POE session. Another POE session that is also running is POE::Component::Server::TCP, which listens for Application B to connect and when it does, will send the JSON data out in an ASCII TCP payload in addition to writing it to the file.

Application B is using POE::Component::Client::TCP to connect to application A and start the data coming in. It also is running a Perl-Tk GUI as a second POE Session to display and help the user analyze the data.

I'm not completely happy with this solution, and would like to explore other options. What would some good choices be?

One obvious problem with this solution is transmitting the data in ASCII. This was convenient because it was how I was saving the data to the output file (so I could look at it with an editor, or feed the file to application B, for debugging purposes). Now I'd like to be a bit smarter by saving/transmitting the data in a more compact form (saving disk space and communications bandwidth). What would be best here (I've used FreezeThaw.pm before - is there something better/different)? The only requirement is that I have to be able to easily reconstitute the original hash containing the data.

Another problem with this solution seems to be that application B runs somewhat slowly. I'm guessing that having the TCP client and Perl-Tk sessions running in the same process is what slows things down. I wonder if things would run faster if the TCP client was in its own process (writing output to a local file), and application B simply used something like POE::Wheel::FollowTail to import the data for the GUI?

Following this train of thought a bit more, I could eliminate all the TCP communication altogether if application A simply wrote the output file, and application B simply tailed the file remotely via ssh or NFS mounted the filesystem and tailed it locally.

Then I started thinking that sending data structures between perl scripts running on different machines might be something so commonly done that there already is a module that will do all this for you. Using all my searching expertise on CPAN, I failed to find anything like this. Am I missing something?

Any help is much appreciated!

-Craig

In reply to Cross-Host Data Transmission by cmv

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.