In other words, after meeting the first <TD ALIGN=LEFT, .+ will match everything up to the last extinfo.cgi in your long string.

To see what I mean, put the first .+ between brackets and print $1.

.+ (and its even more treacherous brother .*) will quickly escape your control if you are not careful. A useful technique to control what gets matched is to indicate the character(s) you don't want: [^>]+, means match anything, except the '>' character, or in other words, until the end of the current HTML tag. It prevents the regex quantifiers to run away.

Dear Perl-Monk,

now I understand whats up with the [^<]+ that has been suggested already. And you are totally right - If I put in brackets whats before the wanted first group, I'll get the whole html-file up to the very last extinfo...

Sadly, when I try to use [^>]+ it still does grab all the content up to the last position. *snip*

/'extinfo\.cgi[^>]+(.+)<\/A.+'status.+>(.+)<\/TD.+nowrap>(.+)<\/TD.+no
+wrap>(.+s)<\/TD>.+'>(\d\/\d)<\/TD>.+'>(.+)<\/TD>/g)
[download]

this will place the whole html file in $1 except for the next groups. How do I have to write this area behind the 'extinfo\.cgi' to make it stop at the > and get the group correctly?

I tried to use

/'extinfo\.cgi[^>]+>(.+)<\/A.+'status.+>(.+)<\/TD.+nowrap>(.+)<\/TD.+n
+owrap>(.+s)<\/TD>.+'>(\d\/\d)<\/TD>.+'>(.+)<\/TD>/g)
[download]

which yield the same result: all of the .html-file inside $1 except for the last group-matches.

I tried to use something like /bla\w{,100}>(.+)<, but this won't match any more. *sigh* a whole working day right now just for making a single RegEx... And I see it comming that I have to insert this "stop at the next whatever" everywhere, because the next .+ between the first () will keep going to the end too, isn't it?

Greetings, a tired Visitor

Please re-read and understand when to use [^>] and when to use [^<]. They are to be used in different situations, as I already told you.

its me again (I truly need an account here), stop pondering about my problem for a while, because I think I figured out how to write the RegEx-Chain of Doom I need to get what I want.

You could try making all your gobbling matches less greedy: Change .+ to .+?. Even better would be [^>]+ for stuff within HTML tags and [^<] for stuff that is supposed to capture (text) content.

Even better would be to use a real HTML parser like for example HTML::TreeBuilder::XPath.

Speaking of which :) htmltreexpather.pl says

//td[@class='statusOdd']
[download]

The links here are tips :)Re: Retrieve select information from HTML, they're examples(for tree-xpath and others)/walkthroughs/tutorials ...

Dear Perl-Monks

I will have a look at the links provided at the page in time, but in the meanwhile I created a counter-example to verify that my codings indeed will yield results in a way I expect them to do.

consider the following file

blabla:(123):falleriefallera
dingdong moep blubb 4711 dingdong
blob))hop((gob))sob((0815))ding
knickknack boing 44

nothing here
blabla:(123):falleriefallera
dingdong moep blubb 471 dingdong
blob))hop((gob))sob((0815))ding
knickknack boing 45

nothing here too
blabla:(1344):falleriefallera
dingdong moep blubb 4711 dingdong
blob))hop((gob))sob((0815))ding
knickknack boing 46

nothing again
blabla:(123):falleriefallera
dingdong moep blubb 4711 dingdong
blob))hop((gob))sob((0825))ding
knickknack boing 47
[download]

access it using the following perl-script:

use strict;
use warnings;

# 1. get file and stuff it into an array
# that what it will be in target code

open FILE, 'target.txt' or die "nope dude: $!";
my @stuff;

while(<FILE>){
    chomp $_;
    push @stuff, $_;
}
print "reading done ";

# 2. make a long line out of it 
# because I still have problems using an array for this :(

my $longline;
foreach my $x (@stuff){
    $longline .= $x;
}

# 3. get all matches and place them in an array array x)

my @super;
while ($longline =~ /\D+(\d+)\D+(\d+)\D+(\d+)\D+(\d+)/g){
    my @sub = ($1, $2, $3, $4);
    push @super, \@sub;
}

# 4. we should have four entries in that @super

print scalar @super, "\n";
[download]

will yield this (at least the debugger think so):

0  ARRAY(0x1f08820)
   0  123
   1  4711
   2  0815
   3  44
1  ARRAY(0x2199678)
   0  123
   1  471
   2  0815
   3  45
2  ARRAY(0x21994e0)
   0  1344
   1  4711
   2  0815
   3  46
3  ARRAY(0x219f128)
   0  123
   1  4711
   2  0825
   3  47
[download]

so it will work in the way I hoped for. IF I ever can create a valid regex for this. But now I'm busy looking into these walktroughs.

By the way; using .+? didn't made the RegEx work, but I don't understand how [^>] should be utilized to help me in my case :( Because... I do find the correct piece of plain text in my file, so how should I include "no >" and "no <" inside?

Greetings, a random visitor

Your example is far more restricted because a character in \D (a non-digit) can never be matched by a character in \d (a digit) and vice-versa.

This is why I suggested that you could use [^<]+ for characters within tags or [^>]+ for characters outside of tags. Both will only match normal characters and not closing (or opening) a tag.

Each greedy match, working left to right (an important point when you have several as in this case), swallows up as many characters as it can while still allowing the match to succeed. So:

$s = 'aaaaaaaaa';
$s =~ /.+(a+)/;    # greedy match
print "$1\n";      # prints "a" (a single 'a')
        
$s = 'aaaaaaaaa';
$s =~ /.+?(a+)/;   # non-greedy match
print "$1\n";      # prints "aaaaaaaa" (all the 'a's)
[download]

So when your pattern is matching too much, start from the end and work your way backwards. Look at what each wildcard pattern (like .+) can match, and think about how to restrict it so it won't match too much. Is there a character that can't appear in it? If so, exclude that character. For instance, if it can't contain any HTML tags, you could use a wildcard like: [^<>]+? That will say, "as few characters as possible, not including angle brackets, while allowing the pattern to match."

Another suggestion: whenever your regex contains forward slashes, use a different regex delimiter so you don't have to backslash the slashes. Even better, learn to use whitespace in your regexes. See how much clearer the second one here is:

$text =~ /$mm\/$dd\/$yy/;   # works, but ugly
$text =~ m|$mm/$dd/$yy|;    # use different delimiter
$text =~ m| $mm            # month
            /
            $dd            # day
            /
            $yy            # year
          |x;              # allow whitespace
[download]

The third example might seem like overkill for such a simple regex, but the longer and more complicated they get, the more these methods help.

Dear Perl-Monks

Thanks a lot for all the help you provided. I finally got it working. For the next weeks I will gladly delegate every RegEx to other workers here, thats sure.

if you wonder how it looks at the end... without the fine art of usung other delimiters, still.

/'extinfo\.cgi[^>]+>([^<]+)<\/A>.{150,190}'status[^>]+>([^<]+)<\/TD><[
+^>]+>([^<]+)<\/TD><[^>]+>([^<]+s)<\/TD><[^>]+>(\d\/\d)<\/TD><[^>]+>([
+^<]+)<\/TD>/g
[download]

I will smooth out this tomorrow, but at least it yield what I was hoping: about 50 Arrays containing the pieces of Information I was looking for.

Thanks again, and until next time :)

... I will gladly delegate every RegEx to other workers ...

That which does not make your brain explode makes you smarter. I understand that your recent experience with the wily regex has been harrowing, but now is not the time to retreat. Rather, consolidate what you have gained and secure some new footholds.

One point you seem not to have grasped is the critical difference between greedy and "lazy" (as I like to call it) matching; e.g., between  .+ and  .+? quantification. Please re-read previous sections in this thread bearing on this.

Perhaps the most important insight to take away is that regexes, much as I love them, are not always the best solution to a problem. Maybe re-consider the advice initially offered by Corion about using a real HTML parser for HTML parsing.

And yes, please do register as a user.

Until next time...

---

Give a man a fish:  <%-(-(-(-<

Maybe re-consider the advice initially offered by Corion about using a real HTML parser for HTML parsing.

Definitely. In 20 years of writing Perl, I've written a lot of long, ugly regexes to pull data out of HTML files as a one-time, quick-and-dirty solution. But I wouldn't count on any of them to be reliable enough to use repeatedly or for automated purposes. For anything reliable, use a module that won't break the day someone changes <TD> to <td> or rearranges a couple of tags.

If the assignment says to do it with a regex, then that's what you do. But in a real-life parsing task, there's usually a better way.

Aaron B.
Available for small or large Perl jobs and *nix system administration; see my home node.