I am currently in a situation where multiple clients need to make data available to one server application. Both the server and client are running on the same machine, so this makes it very easy to handle. My question is: which way is better? I've thought of having the clients append to a flat text file, while the server uses tail -f to get at the data. My second thought was using a database. Here is example structure of my thoughts. First, the database method: 

##################
# Server

my $id = 0;
while (1) {
 # do mysql query: "SELECT FROM msgs WHERE id>?", $id

 for my $msg ($results_of_mysql_query) {
  # do something with this line of data
 }
 sleep 1;
}

##################
# Client:
while (<STDIN>) {
 chomp;

 # do mysql insert: "INSERT INTO msgs VALUES(?,?)", '', $_;
}

[download]

Then let's take a look at the flat text file way of getting this done: 

##################
# Server:

open my $msgs, "| tail -f data.txt" or die "ouch: $!";
while (<$msgs>) {
 # do something with this line of data
}


##################
# Client:
use Fcntl ':flock';

while (<STDIN>) {
 chomp;
 open my $msgs, '>>', 'data.txt' or die "ouch: $!";
 flock $msgs, LOCK_EX;
 print( $msgs, ($_ . "\n") );
 close $msgs;
}

[download]

So out of these two methods, which one strikes you as being more efficient? Or can you think of a different method altogether that beats the pants of both of these? As well, is there any flaw in my logic of the file locking for the flat file method? Will `tail` still be capable of reading the appended data, even if I take an exclusive lock on the file while appending is done? Thanks :)

If the above content is missing any vital points or you feel that any of the information is misleading, incorrect or irrelevant, please feel free to downvote the post. At the same time, please reply to this node or /msg me to inform me as to what is wrong with the post, so that I may update the node to the best of my ability.

In reply to Database vs. tail -f by Coruscate

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.