newlines in Parse::RecDescent

kgoess has asked for the wisdom of the Perl Monks concerning the following question:

This should be a simple problem, but it's stumped me.

I thought Parse::RecDescent would be an easy way to deal with these records, and it's worked fine as long as our customers used the format with a '~' as end-of-record marker, but with just bare newlines it fails miserably.

Can anybody tell me what I'm doing wrong here (in this highly simplified version)? I'm ready to do s/$/~/g, but that's so cheesy :-(

use Parse::RecDescent;
use strict;

my $text_to_parse_with_tildes = <<EOL;
IEA*1*000002669~
IEA*2*000003333~
EOL

my $text_to_parse_plain = <<EOL;
IEA*1*000002669
IEA*2*000003333
EOL

my $grammar = q{

doc: segment(s)
    { $return =1; }

segment: segmentid "*" element(s /[*]/) segment_end
    {  print STDERR "got a $item{segmentid} ($main::segment_end)\n";
       $return =1;  }

segmentid: /[a-zA-Z0-9]+/
    { $return = $item[1]; }
 
element: /[^*~\n]*/
    { $return = $item[1];} 

segment_end: /$main::segment_end/
    { $return = 1;}
    
};


my $parser = new Parse::RecDescent ($grammar) or die "Bad grammar!\n";

#tildes as record-ends, works fine
$main::segment_end  = '~\n';
defined $parser->doc($text_to_parse_with_tildes)  #works fine
   or print "parse failure! bad text!\n";

#plain record-ends, fails miserably
$main::segment_end = '\n';
defined $parser->doc($text_to_parse_plain)  #fails
   or print "parse failure! bad text!\n";
[download]

Comment on newlines in Parse::RecDescent Download Code

Replies are listed 'Best First'.
Re: newlines in Parse::RecDescent by ikegami (Patriarch) on Sep 09, 2005 at 00:49 UTC
Refer to Parse::RecDescent's `<skip>` directive: For the purpose of matching, each terminal in a production is considered to be preceded by a "prefix" - a pattern which must be matched before a token match is attempted. By default, the prefix is optional whitespace (which always matches, at least trivially), but this default may be reset in any production. That's a convoluted way of saying all productions are implicitly prefixed with a `/\s*/` rule. This can be overridden as follows: `... doc: <skip:""> segment(s) { $return =1; } ...` [download] `output ====== got a IEA (~\n) got a IEA (~\n) got a IEA (\n) got a IEA (\n)` [download]	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: newlines in Parse::RecDescent
by ikegami (Patriarch) on Sep 09, 2005 at 00:49 UTC

Refer to Parse::RecDescent's <skip> directive:

For the purpose of matching, each terminal in a production is considered to be preceded by a "prefix" - a pattern which must be matched before a token match is attempted. By default, the prefix is optional whitespace (which always matches, at least trivially), but this default may be reset in any production.

That's a convoluted way of saying all productions are implicitly prefixed with a /\s*/ rule. This can be overridden as follows:

...

doc: <skip:""> segment(s)
    { $return =1; }

...
[download]

output
======
got a IEA (~\n)
got a IEA (~\n)
got a IEA (\n)
got a IEA (\n)
[download]

[reply]
[d/l]
[select]