Popcorn Dave has asked for the wisdom of the Perl Monks concerning the following question:

Fellow monks,

I've got a problematic regex that I hope is something simple that I'm just overlooking.

Given a line of text (and yes that's one contiguous line):

a href="page.cfm?objectid=11933900&method=full&siteid=50144" Costly false alarms /a a href="page.cfm?objectid=11933890&method=full&siteid=50144" Mindless yobs terrorise OAP's /a a href="page.cfm?objectid=11933879&method=full&siteid=50144" Road deaths /a a href="page.cfm?objectid=11933842&method=full&siteid=50144" Twisted porn pervert caged for life /a a href="page.cfm?objectid=11933800&method=full&siteid=50144" Greenbelt homes plan appeal thrown out /a a href="page.cfm?objectid=11933742&method=full&siteid=50144" Youngsters invited to get in the swim /a a href="page.cfm?objectid=11933698&method=full&siteid=50144" Phone mast fears grow /a a href="page.cfm?objectid=11933695&method=full&siteid=50144" Vandals strike at cemetery /a a href="page.cfm?objectid=11928420&method=full&siteid=50144" Windfarm fears are no flights of fancy /a a href="page.cfm?objectid=11928280&method=full&siteid=50144" Mindless vandals wreck headstones /a a href="page.cfm?objectid=11928277&method=full&siteid=50144" Old Firm united to support hospice /a a href="page.cfm?objectid=11928275&method=full&siteid=50144" Family at war over the loss of buses /a a href="page.cfm?objectid=11928273&method=full&siteid=50144" Joining with Jubilations /a a href="page.cfm?objectid=11927986&method=full&siteid=50144" Catch the football fever at coaching days /a a href="page.cfm?objectid=11927859&method=full&siteid=50144" Radio Law ready to hit the airwaves /a
<'s and >'s have been removed to show the text non-html

and a regex of:

m!(<a[^>]*>)(.+</a[^>]*>)!ig

Shouldn't I get $1, $2, $3, etc... until I stop getting matches? All I'm getting at this point is $1 and $2, with $1 having the correct value and $2 having the rest of the string.

Shouldn't the </a[^>]*> match the first occurence of or am I misunderstanding something in my regex?

And yes I *know* I could be using a module to parse this, but this is the exact reason I'm doing it this way -- to better my understanding of regexes.

It also occured to me to split this in to an array and match it that way, but I was unsure on what I could split that on given the number of spaces between tags. Any advice on that would be great too!

Thanks in advance!

Some people fall from grace. I prefer a running start...

Replies are listed 'Best First'.
Re: Regex isn't performing like I think it should
by chromatic (Archbishop) on Jun 12, 2002 at 04:13 UTC
    .+ is greedy. It's so greedy, it matches to the end of the string. Then the regex backtracks until it can match a left angle brace, and finally matches the rest of the regex. (In fact, all of your quantifiers are greedy. You're just fortunate that you provided a negated character class to avoid the trouble.)
Re: Regex isn't performing like I think it should
by jmarshall99 (Acolyte) on Jun 12, 2002 at 04:35 UTC
    Like chromatic said, it is the .+ that is messing you up. You can make that non-greedy by appending a ?. So, your example might become:
    while ( $string =~ m!(<a[^>]*>)(.+?</a[^>]*>)!ig ) { print "$1 \n"; print "$2 \n"; }
    I tried that against a portion of your example string (after replacing the <'s and >'s), and it seemed to work ok. I'm pretty much a newbie, so I imagine there are better ways to do it then I have described. Edit: By the way, it's the "g" option on the regex that makes it work in that "while". I guess when it tries to do a multiple match, it will eventually fall off the end, and return a false. Without the "g", that while statement I used will run forever, as long as the string contains a single match to your regexp.
      Thanks for that! I'm not that far beyond newbie myself :)

      That does work, but it still fails to get anything but the first match even though I'm telling it to do it globally ( or at least I think I am ). That's the part that's still throwing me.

      Some people fall from grace. I prefer a running start...

        Did you try the using "while", as well? I've got this:
        #!/usr/bin/perl -w my $string = '<a href="page.cfm?objectid=11933900&method=full&siteid=5 +0144"> Costly false alarms </a> <a href="page.cfm?objectid=11933890&m +ethod=full&siteid=50144"> Mindless yobs terrorise OAPs </a> <a href=" +page.cfm?objectid=11933879&method=full&siteid=50144"> Road deaths </a +> <a href="page.cfm?objectid=11933842&method=full&siteid=50144"> Twis +ted porn pervert caged for life </a> <a href="page.cfm?objectid=11933 +800&method=full&siteid=50144"> Greenbelt homes plan appeal thrown out + </a>'; while ( $string =~ m!(<a[^>]*>)(.+?</a[^>]*>)!ig ) { print "1st match: $1 \n"; print "2nd match: $2 \n\n"; }
        and when I run it, I get this:
        1st match: <a href="page.cfm?objectid=11933900&method=full&siteid=5014 +4"> 2nd match: Costly false alarms </a> 1st match: <a href="page.cfm?objectid=11933890&method=full&siteid=5014 +4"> 2nd match: Mindless yobs terrorise OAPs </a> 1st match: <a href="page.cfm?objectid=11933879&method=full&siteid=5014 +4"> 2nd match: Road deaths </a> 1st match: <a href="page.cfm?objectid=11933842&method=full&siteid=5014 +4"> 2nd match: Twisted porn pervert caged for life </a> 1st match: <a href="page.cfm?objectid=11933800&method=full&siteid=5014 +4"> 2nd match: Greenbelt homes plan appeal thrown out </a>
        which I think is what you want. I first tried just doing a single "$string =~ m!(<a^>*>)(.+?</a^>*>)!ig" and then printing out $1, $2,...$8, and only got values for the first two. That "while" thing seems a little shady, for the reason I put in my updated post above, but it seems to work in this case.
        What you're running into here is a matter of context. You're evaluating the regex in scalar context, which, when used with the /g modifier, performs a progressive match. This means that every time that regex is evaluated, it starts trying to match immediately after the previous. In other words, it keeps track of it's position in the string and starts where it left off.

        One solution to this is to use a while loop to take advantage of the progressive matching (every time the /g regex is eval'd you get the next values for $1 and $2) as jmarshall99 did. The other is to evaluate your regex in list context to rip out all the matches at once.

        my $string = '<a href="page.cfm?objectid=11933900&method=full&siteid=5 +0144"> Costly false alarms </a> <a href="page.cfm?objectid=11933890&m +ethod=full&siteid=50144"> Mindless yobs terrorise OAPs </a> <a href=" +page.cfm?objectid=11933879&method=full&siteid=50144"> Road deaths </a +> <a href="page.cfm?objectid=11933842&method=full&siteid=50144"> Twis +ted porn pervert caged for life </a> <a href="page.cfm?objectid=11933 +800&method=full&siteid=50144"> Greenbelt homes plan appeal thrown out + </a>'; my @matches = $string =~ m!(<a[^>]*>)(.+?</a[^>]*>)!ig; print "$_\n" foreach @matches;
        Or even using a hash (as long as your a tags don't repeat)...
        my %matches = $string =~ m!(<a[^>]*>)(.+?</a[^>]*>)!ig; foreach my $key (keys %matches) { print "$key\n"; print "$matches{$key}\n\n"; }

        -Bird

      A different (and, I suspect, more efficient) way to handle this would be to use another negated character class instead of the non-greedy modifier:
      while ( $string =~ m!(<a[^>]*>)([^<]+</a[^>]*>)!ig )
      (Personally, I'd make that [^<]* instead of +, just in case you get data with links <right><up><against><each><other>, but that's your call.)
Re: Regex isn't performing like I think it should
by u914 (Pilgrim) on Jun 12, 2002 at 05:40 UTC
    If the goal here is to extract substrings from the long url, and they are wrapped in a consistent set of 'tags' (not neccesarily html tags), then maybe you could look at Text::Balanced

    Not long ago, i asked a question about Text::Balanced, and recieved some really great help, even feedback from the author of the module....

    here's a copy of my code snippet from that page, it looks for all of the 'substrings' that are wrapped in the 'tags' (which can be any arbitrary text, and need not be the same for beginning and ending tags) and strips them out....
    this might be similar to what you're trying to do.

    # find all the URLs from the page contents, rejecting any from bianca @data = extract_multiple( $response->content, [ sub {extract_tagged($_[0], '<a href="http://', '</a>', undef, {reject => ['bianca.com']} ) } ], undef, 1); # loop thru and strip the URL to it's bare address, this is # what's needed to insert into the database for (my $i=0; $i<=$#data; $i++) { my @temp = extract_tagged($data[$i], '<a href="http://', '">', und +ef, undef); $data[$i] = $temp[4]; }
    good luck!