LanX has asked for the wisdom of the Perl Monks concerning the following question:

Hi

I always thought that codefilters are executed before any parsing, but it seems to me that the code already has to be valid before parsing or validation.

Example: The following code tries to use the character X as a new kind of sigil (for motivation please see Re^2: Two more Features Perl 5 Maybe Needs) 

#!/usr/bin/perl
no warnings;
use Filter::Simple;

my $transform = sub {
    s{ X(\w+)\[ }
     {\$$1->\[}xg;
    s{ X(\w+) }
     {\$$1}xg;
    };

FILTER_ONLY    "executable_no_comments" => $transform;


while ($line=<DATA>) {$_.=$line}


if ($_) {
    print "\nbefore:\n",$_;

    &$transform;
    print  "\nafter:\n", $_;

    print "\neval:\n";
    eval $_;
}

__DATA__
Xarr=[1..3];
print "@Xarr";

[download]
this code works as intended: 
before:
Xarr=[1..3];
print "@Xarr";

after:
$arr=[1..3];
print "@$arr";

eval:
1 2 3

[download]
but if I comment out the lines starting with while and __DATA__ to filter the code directly I get: 
Can't modify constant item in scalar assignment at /home/lanx/perl/sig
+il.pl line 29, near "];"
Execution of /home/lanx/perl/sigil.pl aborted due to compilation error
+s.

[download]
line 29 is Xarr=[1..3]; which is of course not valid before transformation.

So is it only possible to use codefilters on syntactically valid code, or did I miss a possibility to filter non-valid code before processing???

Cheers Rolf

Replies are listed 'Best First'.
Re: Are codefilters restricted to valid code?
by Corion (Patriarch) on Dec 22, 2008 at 14:31 UTC

    I'm not sure that Filter::Simple is supposed to work with both, the filtering code and the filtered code in one file. I've split up your code into two files and then changed executable_no_comments to code and now it works :)

    #!/usr/bin/perl
package LanX;
use strict;
use Filter::Simple;

my $transform = sub {
        warn "$_";
    s{ X(\w+)\[ }
     {\$$1->\[}xg;
    s{ X(\w+) }
     {\$$1}xg;
    warn "=> $_";
    $_;
};

FILTER_ONLY "code" => $transform;
#FILTER_ONLY "executable_no_comments" => $transform;

1;

    [download]

    and

    #!/usr/bin/perl
#use strict;
BEGIN {
    require '732082-f.pm';
    LanX->import();
};

Xarr=[1..3];
print "@Xarr";

    [download]
[reply]
[d/l]
[select]
      Thx Max! 8 )

      I got confused with the All-in-one_interface (and my personal x-mas-virus ... cough!!! ; )

      Unfortunately "executable_no_comments" doesn't work, changing to "code" seems neccessary!

      Cheers Rolf

      UPDATE while general codefilters are executed before parsing , most of the FILTER_ONLY options in Filter::Simple rely on valid code (which in general makes sense, but surprises in this example...)

[reply]
Re: Are codefilters always restricted to valid code?
by almut (Canon) on Dec 22, 2008 at 14:33 UTC

    Source code filters are definitely not restricted to valid code. You could, for example, use a source filter to decrypt some encrypted source stream (see Filter::decrypt that comes with the module Filter::Util::Call for a demo implementation, if interested). In other words, the problem lies elsewhere :)

[reply]
[d/l]

Back to Seekers of Perl Wisdom