speedyshady has asked for the wisdom of the Perl Monks concerning the following question:

I posted a question earlier about how to parse a file containing 4Test code with Parse::RecDescent (http://www.perlmonks.org/?node_id=849987), though I'm not really sure how to get this working the way I would like. I've defined my grammar, and ran some evals and it seems to be working fine, but hopefully this isn't all for naught--what I want to do is take arguments from a command line (.txt) and then parse the file. Then I want to print the results to either a new file or instead, overwrite it. I can't seem to find any examples that pertain to my situation. In synopsis I want: 4Test => Perl

Parse::RecDescent is quite difficult to understand fully, but I hope someone can comprehend what I'm trying to do or tell me that I have it completely wrong :-/.

Replies are listed 'Best First'.
Re: Parsing and Translation pt.2
by graff (Chancellor) on Aug 06, 2010 at 22:12 UTC
    I'm not sure I understand what sort of problem you're having (and the code you posted in the reply above doesn't really help -- maybe you could provide some minimal script that actually runs and prints something?) Anyway, just to check... you said:

    ... I ran some evals and it seems to be working fine ... what I want to do is take arguments from a command line (.txt) and then parse the file.

    When you "ran some evals", how did you do that, exactly? Normally, PRD works on a string that you supply to some particular "rule" defined in your grammar. Apparently, in your "evals", you've just been passing numbers -- these get "stringified" by automatically by perl so that string operations can be performed on them.

    In order to pass a string that comes from a file, just read the file contents into a scalar variable, and pass that variable to the parser rule -- e.g.:

    my $input_text; { local $/; # turn on slurp mode within this block open( my $input, "<", $ARGV[0] ); $input_text = <$input>; } $parser->my_first_rule( $input_text );
    Presumably, your grammar will be set up so that your "first" rule will invoke sub-rules for the various expected components of your input text, or will tokenize the text and do something sensible with each token; e.g. based on the snippet of grammar in your reply above, a "first rule" might be something like:
    first_rule : identifier | binops | lbinops | integer | number
    But I haven't used PRD or anything like it in many years, so I'm a little fuzzy on how the grammar definition works. There's bound to be more to it... (update: e.g. you will probably need a rule to handle white space, possibly with different rules for different kinds of white space)
Re: Parsing and Translation pt.2
by toolic (Bishop) on Aug 06, 2010 at 18:04 UTC
    I posted a question earlier about how to parse a file containing 4Test code with Parse::RecDescent
    It is a good idea to provide a link to that node. Is this the one? Parsing and Translation
    Parse::RecDescent is quite difficult to understand fully, but I hope someone can comprehend what I'm trying to do
    There are people here who can advise you, but I think you need to post a small sample of the code you have tried, and where you are having problems.

      I don't want to post the entire code because it is unnecessary. In fact I don't have any problems with the parser I have written. The grammar works fine and my evals all return what I have asked them to. All I want to know/do is read in a file and parse it. I'm having trouble understanding if this is in fact how Parse::RecDescent could potentially be used.

      Here is part of the code:
      use strict; use warnings; use Parse::RecDescent; $::RD_ERRORS = 1; $::RD_WARN = 1; $::RD_HINT = 1; my $grammar = q{ identifier : /[a-z]\w+/ binops : '+' | '-' | '/' | '*' | '%' | '**' lbinops: '!' | '<' | '>' | '>='| '<='| '&&'| '||' | '==' integer: /-\d+/ {print "hello $item[-1]" if $::debugging;} number : /(\d+|\d*\.\d+)/ {print "hello $item[-1]" if $::debuggin +g;} }; #evals: these will evaluate to "1" and print #print "valid number\n" if $parser->number(2); #print "valid integer\n" if $parser->integer(-2);
        All I want to know/do is read in a file and parse it.
        The following code will read an input file, parse using Parse::RecDescent, then write the output to a file. I adapted the example code from Parse-RecDescent-FAQ (Super Search is your friend).

        The input file consists of just a single line: aa

        The output file also consists of just a single line: a

        use strict; use warnings; use Parse::RecDescent; use File::Slurp; my $grammar = 'startrule: ( "a" | "aa" ) "a"'; my $parser = Parse::RecDescent->new($grammar); my $text = read_file('input.txt'); write_file('output.txt', $parser->startrule($text), "\n");

        Hopefully, you can apply this trivial example to your code. (Keep in mind that this is the 1st time I've ever used Parse::RecDescent.)

        So the problem is that you don't know how to read a file?
        my $file; { open(my $fh, '<', ...) or die ...; local $/; $file = <$fh>; }

        But it also seems that your grammar isn't complete. You've defined some basic blocks, but not how they fit together. If you just want a tokenizer, you can use something like

        use strict; use warnings; use Parse::RecDescent; $::RD_ERRORS = 1; $::RD_WARN = 1; $::RD_HINT = 1; my $grammar = <<'__EOI__'; { use strict; use warnings; } tokenize : token(s?) /\Z/ { $item[1] } token : ident | binops | lbinops | number | integer | /./ { [ skip => $items[1] ] } ident : /[a-zA-Z][a-zA-Z0-9_]*/ { [ @items ] } binops : /\*\*|[+\-\/*%]/ { [ @items ] } lbinops : />=?|<=?|!|&&|\|\||==/ { [ @items ] } number : /-?[0-9]*\.[0-9]+/ { [ @items ] } integer : /-?[0-9]/ { [ @items ] } __EOI__ ... my @tokens = $parser->tokenize($file);