in reply to HTTP Requests and Threading

G'day carlriz,

Welcome to the monastery.

In the main, this looks like a well-written piece of code. Were you just looking for comments on the code or are you experiencing some difficulties (e.g. something doesn't work as expected, excessive run time, chews up lots of memory, etc.)?

If the 'x50' (in my @URLs = ...) is related to the THREADS constant, then 'x THREADS' would be a better choice.

I'd probably set the timeout value as a constant (near the top of the script) and keep comments relating to that value there. The current comment refers to testing so presumably that's intended to change during the development cycle. Consider the chances of forgetting to search through the script for $ua->timeout(5) and changing it, against the chances of not spotting TIMEOUT => 5, # for testing near the top; you also won't have to search for $ua->timeout(TIMEOUT) (however many times it occurs) as it won't need modification.

Overall, the layout looks very good and is easy to read. I'd aim to keep lines <= 80 characters (others have different preferences: 72 and 78 characters are common). When lines start to get much longer than this, readability suffers. Consider the readability of one of your lines and the suggested alternative following it: 

my %query_hash = (action => 'submit or status', name => 'chris',  outn
+ame => 'chris', 'content-type' => 'application/json');

[download]
my %query_hash = (
    action         => 'submit or status',
    name           => 'chris',
    outname        => 'chris',
    'content-type' => 'application/json'
);

[download]

Finally, of minor importance and I suspect this may be an artefact of your editor, you have a large amount of extraneous whitespace after many of you lines. Because this is wrapping in my browser (and I suspect this will be true for many others), I actually see your intended 

#########
# Main  #
#########

[download]

as

#########
+
# Main  #
+
#########
+

[download]

There's many examples of that wrapping issue throughout the code you posted.

-- Ken

Replies are listed 'Best First'.
Re^2: HTTP Requests and Threading
by carlriz (Beadle) on Mar 06, 2014 at 19:43 UTC

    Hi ken, Thanks for the thorough message. I am having difficulties with outputting my data to stdout, in the format that I wanted. The way I wrote it is when the individual threads complete, they execute this code: 

    
 if( $response->is_success ) {

            print"\n\n", $counter_thread++,"th :", "Dumped Data Return
+ed:\n\n ", Dumper($response->decoded_content), "\n" ;

            print "Response code:  ", $response->code, "\n",
            "HTTP Response Message: ",
            $response->message, " ", "\n";

           } else {
             print STDERR $response->status_line, "\n";
          }

    [download]

    However, they come out in unsorted order. I am trying to print each thread in order with time elapsed for each thread, decoded_content in a neat way. As an example, the variable '$counter_thread' would print like 2,5,8, 10,7, etc. to stdout. Is this a threading issue? What is the best way to go about this? Chris

[reply]
[d/l]

      Your threads run concurrently and will take different times to complete depending upon a bunch of factors -- the remote server's workload; distance and latency of the link between your machine and those servers; the size of the page returned; etc. -- so the order in which they finish will be unrelated to the order that you start them. That is normal.

      If you want to control the order of the output, you will need to accumulate the results and arrange to output them in your desired order.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

        That makes sense.

[reply]
[reply]

In Section Seekers of Perl Wisdom