Hi,  Preface/disclaimer, I'm using someone else's script.
 So, although I can understand the perl, I don't know the http stuff. I'm using the script to send a text file one-line-at-a-time, converting each line of text into an http query.
 The problem I have is that the script as it stands creates a bottleneck such that even if I have multiple http collectors (for lack of a real term), the process of sending out the whole file and collecting the reponses takes as long as from a single machine.
 My question is this, is this inevitable and a consequence of the way that a script transmits then waits for a response before sending the next transmission, or is there a way to hack the script to be able to send off multiple queries ?
 One thought that occurs to me is that there is problem of ordering that is presently solved by waiting (I think) for a response so that each query is paired with its response. This is essential to my task, so perhaps there is a subsidiary question as to whtether multiple queries can be sent out and their responses collected and paired with them in the correct manner.
My apologies for the wordiness and lack of knowledge in this post, and please don't feel that I'm asking for a solution, just whether or not what I desire is possible. The code is as follows:
<CODE> #!/usr/bin/perl
#
# Script: httpQueriesLoop.pl
#
# Purpose: Continuously send queries stored in a text file
# as http queries
#
use Getopt::Std;
use Shell;
use URI::Escape;
$TMP_FILE = "_query.txt";
# get command-line options
if (! getopts('f:hm:v')) {
displayUsage();
}
if (defined($opt_f)) {
$FILENAME = $opt_f;
}
else {
displayUsage();
}
if (defined($opt_h)) {
displayUsage();
}
if (defined($opt_m)) {
$HOSTNAME = $opt_m;
}
else {
$HOSTNAME = "127.0.0.1";
}
if (defined($opt_v)) {
$VERBOSE = 1;
}
else {
$VERBOSE = 0;
}
# process text file containing queries; 1 query per line.
open (FILEHANDLE, $FILENAME) or die "Cannot open
\"$FILENAME\" file.\n";
if ($VERBOSE) { print "\tProcessing queries in
$FILENAME...\n" };
$counter = 1;
while (1) {
while (<FILEHANDLE>) {
chomp; # no newline
s/#.*//; # no comments
s/^\s+//; # no leading white
s/\s+$//; # no trailing white
next unless length; # anything left?
if ($VERBOSE) { print "Query = $_\n"};
my $escapedQuery = uri_escape($_);
# if ($VERBOSE) { print " Escaped Query = $escapedQuery\n"};
`echo content=$escapedQuery > $TMP_FILE`;
$req = "lwp-request -m POST http://$HOSTNAME:8080/mydestination/connect < $TMP_FILE";
# if ($VERBOSE) { print $req . "\n" };
system ("$req");
print "message #$counter\n";
$counter++;
}
close(FILEHANDLE);
}
# display usage
sub displayUsage {
die <<EOF
Usage: httpQueriesLoop -f "textfile" -m "hostname" -v
Arguments:
-f textfile containing queries
-h display this help message
-m host machine running connector; defaults to localhost
-v verbose
Example: httpQueriesLoop -f "c:/temp/foo.txt" - m "128.138.0.20" -v
EOF
} # sub displayUsage

In reply to distributing http queries by oconnelm

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.