Many of the templating systems can do the replacement for you. Why do you want to talk to a separate Perl process at all if there is eval?

# First stage:

sub process_perl {
    my( $code ) = @_;
    my $res = eval $code;
    my $err = $@;

    warn "Whoops. Caught error [$err] for code [$code]"
        if $err;

    return $res;
};

$template =~ s!<%([^%]+)%>!process_perl("$1")!gse;
[download]

If you want/need to do a second round of replacements then, do it afterwards:

$template =~ s!\binclude\b ([^\s]+)!process_include("$1")!gse;
[download]

The issue is that the file syntax is *not* Perl, i.e. there can be things like this:

CHAIN<%=$chain_num%>_EN [<%=$lsb%>:<%=$lsb++%>] = 1'h0;
[download]

Even the format spec document suggests this approach "wrap all non-Perl chunks with prints, leaving the code snippets as is, and let Perl evaluate the whole thing".

The reason a simple eval() doesn't work is because there is Perl variable state that needs to be honored across the entire file (e.g. $lsb in the example above is set to 0 at the start and incremented a number of times through the file. Hence, we will need a full Perl process to evaluate the whole file.

In any case, after understanding the REPL issue, I broke up the processing into 2 sections:

* send the entire file to the Perl child process, and close the child's input filehandle

* then read back the generated output in a while loop

* if we encounter an "include" statement, then recursively restart the process with the new file.

This works perfectly. Thanks for all the suggestions.

Pankaj

Hi StoneLeopard,

The reason a simple eval() doesn't work is because there is Perl variable state that needs to be honored across the entire file (e.g. $lsb in the example above is set to 0 at the start and incremented a number of times through the file. Hence, we will need a full Perl process to evaluate the whole file.

I don't follow this argument as a reason for needing a separate process. The state of package variables (those declared with our, or if no strict 'vars' is in effect, any undeclared variables) is honored across evals:

$ perl -le 'eval q{ print ++$x }; eval q{ print ++$x }'
1
2
[download]

If you had said that you don't want code from evaling one file to affect code from another evaled file, that would be a slightly different story. There are a few Perl modules, pragmas and variables which affect everything in the interpreter process, so if those are used, I would agree that yes, the best way to go is starting up a new interpreter for evaluating each file. But other than that, most Perl code can be compartmentalized in a package just fine. A package within its own block would hold lexicals, package variables, subroutine definitions, pragmas with lexical scopes such as strict and warnings, and in most cases subroutines imported from used modules, and all those would be kept separate from other packages. And special variables can and should be localized.

The suggested approach "wrap all non-Perl chunks with prints, leaving the code snippets as is, and let Perl evaluate the whole thing" makes sense to me. And just to show that it can be done all within a single Perl process with eval, here's a simple implementation of exactly that. Doing a second round of replacements, as Corion said, is just a matter of applying the same idea a second time.

File a.txt:

CHAIN<%=$chain_num%>_EN [<%=$lsb%>:<%=$lsb++%>] = 1'h0;
<% use File::Spec::Functions qw/catfile/ %>
[download]

File b.txt:

<%$lsb++%>
<% for my $x (0..2) { %>
Hello, World Nr. <%=$x%> (LSB=<%=$lsb%>)
<% } warn "test warning" %>
$this is not $Perl
<% # note catfile() not available here %>
[download]

The script:

Read more... (2 kB)

Output of running perl script.pl a.txt b.txt:

>>>>> Output:
CHAIN_EN [:0] = 1'h0;
<<<<<
test warning at Generated2 line 8.
>>>>> Output:
Hello, World Nr. 0 (LSB=1)
Hello, World Nr. 1 (LSB=1)
Hello, World Nr. 2 (LSB=1)
$this is not $Perl
<<<<<
[download]

Note how the two files each have their own $lsb variable.

Of course, I've just re-invented the wheel (for fun, so it's okay :-) ). See for example Embperl.

Hope this helps,
-- Hauke D

Update: Added info on special variables.

Update 2018-03-10: Posted a modified version of the above as this StackOverflow answer.