Hena has asked for the wisdom of the Perl Monks concerning the following question:

Hello to all monks,

I'm currently parsing a string with defined format (tag[value]) like this: 

while (m/(\w{1,2})\[(.+?)\]/g) {
  push(@{$hash{$1}},$2);
}

[download]
Now the value ($2 in code) should be allowed to have regex pattern within it. This causes the obvious problem of have tag like T[[pP]attern]. So i could go through the string one char at a time and calculate openings ([) so that i get the tag and value pairs, but it doens't look like perl (more like c).

So the question becomes, how to do that with regex? I cant figure out.

I extended it so that it wont hit this problem so easily, but it's just pushing the problem. 
 while (m/(\w{1,2})\[(.+?)\](?=(?:\w{1,2}\[)|\z)/g) {
  push(@{$hash{$1}},$2);
}

[download]
TIA

Edit by tye, escape [

Replies are listed 'Best First'.
Re: Regex dynamics
by Fletch (Bishop) on Feb 10, 2004 at 15:35 UTC

    Sounds like you want Regexp::Common and it's  $RE{balanced}{-parens=>'[]'}.

[reply]
[d/l]
Re: Regex dynamics
by halley (Prior) on Feb 10, 2004 at 15:38 UTC
    Look at CPAN's docs for Text::Balanced, especially the quotelike parsing feature. This can read any of Perl's qq{} s/// m// tr/// formats.

    --
    [ e d @ h a l l e y . c c ]

[reply]
[d/l]
Re: Regex dynamics
by ysth (Canon) on Feb 10, 2004 at 15:59 UTC
    (Update: I see responses assuming this is a balanced delimiter question; it is not, it's more a parsing perl question. The brackets in a regex don't necessarily balance.)

    Taking just the [] part into consideration, what you want to match is going to be zero or more (one or more?): of either a series of other characters or a [...] pair. So at first blush, something like this: 

    m/\[                 # initial [
  (?:
     [^][]+          # non-bracket parts
  |
     \[somethinginbrackets\]  # something in brackets
  )*                 # repeated
\]                   # final ]
/x

    [download]
    But (as that regex itself shows) the something in brackets itself could be a little complicated to parse.

    First, note that if you allow interpolated variables in your regex, the problem is a lot more complicated. $foo[bar] in a regex could be either the scalar $foo followed by a character class, or an interpolated array element. Perl uses a guessing function that gives different weights to characters found in the "bar" part to decide which (and doesn't always get it "right"). If "bar" is an array index, it will be arbitrary perl code. See "Gory details of parsing quoted constructs" in perlop for how to locate the ending ]. Also, you need to allow for interpolated aribitrary code: @{[foo(),"]"]}. That's not a matter of matching square brackets; the gory details rules will help find the ending "}".

    Assuming you don't need to handle interpolation in the regex, you still have to deal with []], [x[:alnum:]y] (vs. [x[:alnum]), [[\]], and similar. See "Version 8 Regular Expressions" and the POSIX character class sections in perlre. (Update: previous sentence unmangled.)

    Update: all things considered, you'd do yourself a great favor requiring your value to be quoted with "" or something else (with literal " in the value requiring a backslash) when the value has a ]. Or just require all brackets in the value to be backslashed. Then it's just a matter of matching m/\[ (?: [^][]+  |  \\. )* \]/x.

    Update: ++halley; I'm going to look at Text::Balanced.

[reply]
[d/l]
[select]
Re: Regex dynamics
by broquaint (Abbot) on Feb 10, 2004 at 15:41 UTC
    ... or just use Regexp::Common e.g 
    while( /(\w{1,2})$RE{balanced}{-parens=>"[]"}{-keep}/ ) {
  push @{ $hash{$1} }, substr($2, 1, length($2) - 2);
}

    [download]
    HTH

    _________
    broquaint

[reply]
[d/l]
Re: Regex dynamics
by borisz (Canon) on Feb 10, 2004 at 15:51 UTC 
    #!/usr/bin/perl
my $q;
$q = qr/(?:\[(?:(?>[^\[\]]+)|(??{$q}))*\])/;

while ( defined( $_ = <DATA> ) ) {
  while (/(\w{1,2})($q)/g) {
    print $1, $2, $/;
    #push(@{$hash{$1}},$2);
  }
}
__DATA__
T[[pP]attern]
X[]
X[xx[xxx[xx]]]

    [download]
    Boris
[reply]
[d/l]
      This is indeed what i wanted. Well it matches the [] as well, but that i can live with no problem :). Thank you.
[reply]
[d/l]
        I did not read carefull enough. Try this line. 
        my $q;
$q = qr/(?:\[(?:(?>[^\[\]]+)|(??{$q}))+\])/;

        [download]
        Boris
[reply]
[d/l]
Re: Regex dynamics
by graff (Chancellor) on Feb 11, 2004 at 01:57 UTC
    Forgive me for being impertinent, but whose idea was it to define a format like this?

    If you have control over that aspect of the design, and if it's not too late to think it over and try again, I would certainly recommend that you just change the design to something that's easier to handle. Of course, if the values of tags can really be any sort of Perl regex, then it might be challenging to come up with something really bullet-proof. Even so, a format like "tag:=:value" is bound to avoid a lot of problems, because you'll have a delimiter that is likely to be distinctive and unambiguous in the vast majority of cases.

[reply]
[d/l]

Back to Seekers of Perl Wisdom