in reply to Re^4: Forking Multiple Threads
in thread Forking Multiple Threads

Bumcombe!!

If I have, as you yourself have said, “2,000 SAAS servers from which to retrieve data, that takes each of them 10 seconds to prepare and return,” then I most-emphatically don’t require “an equal number of my processor threads to be sitting around, each of them waiting for one of those SAAS servers to answer me.“

If I am waiting to hear back from “2,000 SAAS servers,” then the one thing that I can absolutely count on is that:   it will not be the case that all 2,000 of them will reply to me within precisely the same millisecond.   There is, after all, one TCP/IP interface card on this box, therefore all of the datagrams will arrive one at a time.   Many, many milliseconds apart.   Therefore, it is entirely to be expected that just one process, on my end, will be able to handle the 2,000 separate responses that will arrive.

Y’know, BrowserUK, I am afraid that these days you spend so much time opposing what I have to say, on the assumption that during the course of 30 years in this business that I have learned absolutely nothing in the way of pragmatic and practical and worldly experience, that you seem to spend almost no time listening to whatever it is that I have to say.   And maybe, (durst I say it?) you really ought to “meditate” on that, for a little-biddy while.   It might just do you some good.   Because, quite frankly, I do not have the time, nor do I have the slightest inclination, to quarrel with you further on these points.

Good day, sir.

Replies are listed 'Best First'.
Re^6: Forking Multiple Threads
by BrowserUk (Patriarch) on Feb 09, 2012 at 04:14 UTC
    Therefore, it is entirely to be expected that just one process, on my end, will be able to handle the 2,000 separate responses that will arrive.

    Oh yeh! Bollocks! Here is your favoured "select loop" (not implemented by me, but rather the best CPAN has to offer to ensure fairness), processing those same 2040 urls that 64 threads dealt with in 88 seconds:

    #! perl -slw
use strict;
use Time::HiRes qw[ time ];
use LWP::Parallel::Useragent;
use HTTP::Request;

my $start = time;

my $pua = LWP::Parallel::UserAgent->new();
$pua->timeout( 10 );

$pua->register(  HTTP::Request->new( 'HEAD', "http://$_" ) ) while <>;

my $entries = $pua->wait;

printf "Took %.6f seconds\n", time - $start;


__END__
c:\test>pua-head-urls urls.list
Took 1333.616000 seconds

    [download]

    Yup! One thousand, three hundred and thirty three point six one six seconds.

    For all your claims of 30 years of experience, and fine protestations, you know nothing. You just regurgitate what you think you've read as authoritative. And usually get it completely wrong.

    Can you do better than the author of LWP::Parallel::UserAgent? I'm betting you will not even try. Just like you never have.

    BTW: I'm not arguing with you. I'm simply telling you that you're wrong. The only counter-argument is proof. Ie. Runnable code that supports your erroneous statements. Ante up or shut up.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?
[reply]
[d/l]

In Section Seekers of Perl Wisdom