Hi all, I have a kind of webcrawler that's using LWP::UserAgent to download large numbers of webpages, basically using 
my $ua = LWP::UserAgent->new(ssl_opts => { SSL_verify_mode => 'SSL_VER
+IFY_NONE'},);
$ua->timeout(45);
...
my $request = HTTP::Request->new(GET => $trg);
$request->protocol('HTTP/1.0');
$request->header('Accept' => '*/*');
$request->header('Connection' => 'Close');
...
my $resp = $ua->request($request);
...

[download]
Results are written into a database. Now my question is if there's a way to find out what local port was used for the connection. I don't need to set the port, but would like to know afterwards what port was chosen.

Background: I need some way to link connections the crawler makes in a unique way with data that a sniffer collects at the same time (which implements suricata rules and a passive SSL collector). As the crawler makes about 20 queries per second, the timestamp alone is not sufficient. As SSL connections are also used, I can't use the URL (because the sniffer doesn't see them). The destination IP would be an alternative, but might not be unique (the crawler migth access the same IP one after another for different URLs). So the local port number would be a good option, together with the timestamp of course. But for this to work, I must be able to figure out the port number the crawler uses (plus prevent shared connections, hence the Connection: close header). I could also try to set the local port in advance (I did read a posting that shows a way to do it), but that can cause errors (double used ports).

One way would be if I could somehow access the socket that LWP used/will use. Any suggestions are highly welcome :-)

Thanks, Andy

In reply to LWP: How to find out local port number? by klaymen

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.