in reply to Help with regex for complicated key=value string

use strict;
use warnings;


sub dequote {
   my ($s) = @_;
   if ($s =~ /^"/) {
      $s =~ s/^"//;
      $s =~ s/"$//;
   } else {
      $s =~ s/\\([\\,])/$1/g;
   }
   return $s;
}


sub parse {
   my @terms;

   for ($_[0]) {
      my $key = /\G ( [^=,]+ ) = /xgc ? $1 : undef;

      my $val = /\G
         (
            " [^"]+ "
         |
            (?: [^\\,] | \\[\\,] )*
         )
      /xgc && dequote("$1");

      push @terms, [ $key, $val ];

      /\G $ /xgc
         and last;

      /\G , /xgc
         or die("Expected comma at pos ", pos(), "\n");

      redo;
   }

   return @terms;
}


# ---


use Data::Dumper qw( Dumper );

my $str = join ',', <<'__EOI__' =~ /.+/g;
key=value\,value
key=\\
key="value,value"
value
__EOI__

my @terms = parse($str);

local $Data::Dumper::Indent = 1;
local $Data::Dumper::Terse  = 1;
local $Data::Dumper::Useqq  = 1;
print(Dumper(\@terms), "\n");

[download]

The above assumes slashes can also be escaped.
The above otherwise assumes everything you didn't specify is disallowed.

The following might be better to get the value, but differs from what you asked: 

sub dequote {
   my ($s) = @_;
   if ($s =~ /^"/) {
      $s =~ s/^"//;
      $s =~ s/"$//;
   }
   $s =~ s/\\(.)/$1/sg;
   return $s;
}

      my $val = /\G
         (
            " (?: [^\\"] | \\. )* "
         |
            (?: [^\\,] | \\. )*
         )
      /xsgc && dequote("$1");

[download]

Update: Fixed to use double quotes instead of single quotes. Misread.
Update: Remove "or die("Expected value at pos ", pos(), "\n")" after the value extraction match. It'll never be executed since the pattern can match zero characters.

Replies are listed 'Best First'.
Re^2: Help with regex for complicated key=value string
by mscharrer (Hermit) on Oct 29, 2008 at 19:22 UTC
    Thanks ikegami, using \G is a good idea. Thanks for all the effort.

    I also coded an simple parser in the meantime, which seems to do the job for me but without doing strict error-checking like yours. It also only splits the string on the commas, not yet the keys and values. Because only alphanumerical keys are allowed anyway (I didn't mentioned that before, sorry) there is no need to look for escaped '=', etc., i.e. everything which doesn't start with /\s*[\w_]+=/ is taken as a single key-less value. 

    #!/usr/bin/perl
use strict;
use warnings;

sub cskwparser {
    my $string = shift;

    my $esc   = 0;
    my $quote = 0;
    my @args;
    my $narg = 0;

  CHAR:
    foreach my $char ( split //, $string ) {
        if ($esc) {
            $esc = 0;
        }
        elsif ( $char eq '"' ) {
            $quote = !$quote;
        }
        elsif ( $char eq '\\' ) {
            $esc = 1;
        }
        elsif ( $char eq ',' and not $quote ) {
            $narg++;
            next CHAR;
        }
        $args[$narg] .= $char;
    }

    return @args;
}

# Test loop:
local $, = '|';
while (<>) {
    print cskwparser $_;
}

    [download]
[reply]
[d/l]
[select]

      $esc is never used in your code, so that means "\\," is not handled yet.

      but without doing strict error-checking like yours

      It's not really. It only checks for 'key=,...' (key with no value) and 'v"alu"e' (invalid quoting).

      Because only alphanumerical keys are allowed anyway (I didn't mentioned that before, sorry) there is no need to look for escaped '=', etc., i.e. everything which doesn't start with /\s*[\w_]+=/ is taken as a single key-less value.

      That's the same strategy I used, but I didn't limit to alphanum chars. To do so, change
      /\G ( [^=,]+ ) = /xgc && ( $key = $1 );
      to
      /\G ( [a-zA-Z0-9]+ ) = /xgc && ( $key = $1 );

[reply]
[d/l]
[select]
        $esc is used: 
                if ($esc) {
            $esc = 0;
        }
...
        elsif ( $char eq '\\' ) {
            $esc = 1;
        }

        [download]
        and "\\," is handled quite well in my tests.

        Do I miss here something?

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom