UPDATE: Seems that the data I was testing for landed right on a buffer chunk boundary. Not good. Let that be a lesson to all us part timers - boundaries suck!
====================================================
Recently, I had to drag my old perl knowledge up from the bottom of my brain to do a little db creation work.

I am taking a web page from our local chamber of commerce, where they give an alphabetical listing of their members, and sucking it in with Net::HTTP. I then cycle through the $buffer looking for occurances of "ID=XXXX". Those are links to "more info" on each company. Using the built-up array of IDs, I then pull each companies individual data.

Anyhow, to make a long story short (too late, right?) one business, serial number 3975, is always skipped!!! the regex $buf =~ /ID=([0-9]+)/ seems to think 3975 doesn't match! The only answer I have found is to first write the main web page to a file, then read in the file. THEN it matches.

Here is the code I am/was using:

#!/usr/bin/perl

use strict;
use Net::HTTP;
use HTML::Strip;
use LWP::Simple;

my $DOMAIN="www.fentonchamber.org";
my $MAIN_LIST="AlphabeticalListing.asp";
my $HOMEDIR="/home/jrobiso2/Documents/CDS/Chamber/";
my $list_file="/tmp/AlphabeticalListing.html";
my @listing;

### Get initial listing of data from the main page.

open(SRC, "+>$list_file") or die "Cannot open file: $!\n";

my $http = Net::HTTP->new(Host => $DOMAIN) || die $!;
$http->keep_alive;
$http->write_request(GET => "/$MAIN_LIST", 'User Agent' => "Mozilla/5.
+0");
my($code, $mess, %h) = $http->read_response_headers;

## Build the listing of company numbers from the javascript window.ope
+n
## calls inside the main listing html page.

while (1) {
    my $buf;
    my $n = $http->read_entity_body($buf, 1024);
    die "read failed: $!" unless defined $n;
    last unless $n;
#   if ($buf =~ /ID=([0-9]+)/) {  #  OLD CODE
#       push @listing, $1;        #  THAT FAILED
#   }                             #
    print SRC $buf;
}

close(SRC);

open(SRC, "<$list_file") || die "Cannot open file: $!\n";

while (<SRC>) {
    if (/ID=([0-9]+)/) {
        push @listing, $1;
    }
}

@listing = sort @listing;

[download]
From the code above you can see the actual site and page I am trying to steal from. If anyone can enlighten me as to what is wrong (what I have done wrong) I would greatly appreciate it, as this has taken 3 hours of my time and made me feel very stupid. I am using perl 5.8.6.


What does this little button do . .<Click>; "USER HAS SIGNED OFF FOR THE DAY"

In reply to Strange regex behavior - beware chunk boundaries! by tame1

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.