OK, I've got this working pretty well. I found Net::Pcap::stats which let me see received and dropped packets and found that the problem was indeed dropped packets. I then switched from 
$sniffer->run(); # uses the "best" default device

[download]
to feeding Sniffer from my own extremely simple Net::Pcap loop. 
     my $err = '';
     my $dev = Net::Pcap::pcap_lookupdev(\$err);  # find a device
 
     # open the device for live listening
     my $pcap = Net::Pcap::pcap_open_live( $dev, 4096, 0, 0, \$err);
     Net::Pcap::pcap_loop( $pcap, -1, \&process_pkt, "user data");
    my %stats;
    $stats{ps_drop}=0;
 sub process_pkt {
    my ($user_data,$hdr,$pkt)=@_;
   Net::Pcap::stats( $pcap,\%stats ) ;
   print "$stats{ps_drop} pkts drpd, $stats{ps_recv} pkts rcvd.\n";
     $sniffer->handle_eth_packet($pkt);
     }

[download]
to try bypassing the grabber in Sniffer::HTTP. I found that I could go for hours without dropping a single packet. I then started looking at the code in sub run in Sniffer/HTTP.pm and the only difference I could see (read understand) was that I had set snaplen to 4096 in the creation of my capture device. I happen to know that what I am looking at is going to be smaller than that. I then changed only that in the Sniffer/HTTP.pm code and now I can use the use the run method and not get dropped packets.

Now I know that you wrote this to cover all reasonable scenarios hence the big number. But what I don't understand is that since snaplen is only supposed to be an upper limit and if the incoming packet is only 1440 bytes the 128000 shouldn't even come into play, right? So why does dropping it to 4096 solve my dropped packet problem?

I realize that this is an issue with Net::Pcap but I'm sure you know more about Net::Pcap than I do and I would like to try and understand how and why what I did seems to have fixed this issue.

Thanks for all your help and putting up with me.

In reply to Re^7: Sniffer::HTTP problem with timeout by ponley
in thread Sniffer::HTTP problem with timeout by ponley

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.