Hello fellow monks,

I use the following snippet to get the number of currently used netscape proxy server processes from a list of machines. For this purpose, one normally uses a browser and a special URL to display a kind of status table. Now the code works fine, but one question appeared while writing it.

The problem was, that the response that the machines (=the netscape proxy server admin process) sends, never 'ends' or to be more precise, the user agent never stops reading data. It seems to be caused by the content type, which is send by the server as

multipart/x-mixed-replace; boundary=THIS_STRING_NEVER_HAPPENS.

The browser refreshes the page after a few seconds on and on and on but also never really ends getting data.
I worked around it by reading only 5000 bytes and then snipping the text out between two boundaries and then processing it.

I wondered if there was a more elegant way to do it.

Any input very appreciated.

Bjoern

_____
All good things come to those who wait.
--Sam

use strict;
use warnings;
use diagnostics;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use IO::Handle;

my $Proxy_Username='admin';
my $Proxy_Password='adminpw';
my @Proxies = (
        "http://proxy1:23651/proxy-proxy1-proxy/bin/sitemon?doit HTTP/
+1.0",
        "http://proxy2:23651/proxy-proxy2-proxy/bin/sitemon?doit HTTP/
+1.0", 
        "http://proxy3:23651/proxy-proxy3-proxy/bin/sitemon?doit HTTP/
+1.0",
        "http://proxy4:23651/proxy-proxy4-proxy/bin/sitemon?doit HTTP/
+1.0");

STDERR->autoflush(1);  
STDOUT->autoflush(1);  

while (1==1) {
my $ReadableTime=localtime(time());
print $ReadableTime," --> ";
foreach (@Proxies) {
    my $URL=$_; 
    my $UA = LWP::UserAgent->new(); # create new user agent
    $UA->agent("Mozilla/4.7 [en] (WinNT; I)"); # needs to be set like 
+this to get an answer from the proxy server 
    $UA->timeout(15); # give the user agent some input
    $UA->max_size(5000); # more input
    my $Request = HTTP::Request->new(GET => $URL); # create new reques
+t
    $Request->referer("http://wizard.yellowbrick.oz"); # perplex the l
+og analysers (stolen code)
    $Request->authorization_basic($Proxy_Username,$Proxy_Password); # 
+some more input for the request
    my $Response = $UA->request($Request); # off goes the request
    if ($Response->is_error()) { # ups, some error here
        print "something wrong happened contacting $URL....\n";
        } 
    else { # everything went ok
        my $OffWeGo=0;
        my $Content = $Response->content();        
        my @ContentArray; # original array
        my @NewContentArray; # text between two boundaries
        @ContentArray=split(/\n/,$Content); # bring on the lines
        foreach (@ContentArray) {
            if (($_=~/--THIS_STRING_NEVER_HAPPENS/) && ($OffWeGo==0)) 
+{ $OffWeGo=1; next; }
            if (($_=~/--THIS_STRING_NEVER_HAPPENS/) && ($OffWeGo==1)) 
+{ $OffWeGo=0; last;}    
            if ($OffWeGo==1) {
                push (@NewContentArray,$_);
            }
        }
        if ($NewContentArray[14]=~/(>\s)(\d*)(\s<)/) {print $2," "; } 
+# get number of active processes
    }
}
print "\n";
sleep (60);
}

[download]

Edit kudra, 2002-09-11 Added a READMORE before the code

In reply to Question concerning HTTP::Request and LWP::UserAgent by Bjoern

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.