howto parse (or determining end) of a line of perl

perl-diddler has asked for the wisdom of the Perl Monks concerning the following question:

I have a little calc I wrote in perl that really just provides and eval & print loop for the user.

I have made changes over the years, but one of the things that has always bothered me is, if I want to add a complex expression -- anything that is multi-lined, how to get my input loop to know when it needs more input OR when it doesn't (vs. _could_ take more input).

My 'semi-model' for that would be something like bash, where if you type 'the beginning of a control structure, bash will change to a different prompt to indicate it wants more input.

How might I do the same in my eval/print loop?

As it is now, I can define functions on 1 line, for example, but there is no easy way to extend that to more than one line.

I could force the use of an 'extend char', like backslash at the end of line -- but in bash, those are only needed if it is ambiguous -- i.e. if the line is already well formed, you need to enter '\' to tell bash to keep parsing. Ex. (using 'home>' as normal prompt):

home> int a=1+1\
> +2; echo $a
4
[download]

You can't enter partial *expressions* in bash and have it "auto continue" (that I know of). I.e.

home> a=1+<cr>
-bash: 1+: syntax error: operand expected (error token is "+")
[download]

But I could enter a '\' at the end of line and continue it as I did above.

Where bash works to auto-detect is in its control structures (or like if a quote is still open). Ex:

home> for ((i=0;i<10;++i)); then<cr>
> [...]
[download]

On the 2nd line, it doesn't display the normal prompt, but a single greater than sign. How could I get my input/eval loop to get feedback from perl that I'm in the middle of a similar structure and change the prompt and not try to eval it?

-------------

Clarification: how can I do that w/o writing an entire perl parser? ;-)

Comment on howto parse (or determining end) of a line of perl Select or Download Code

Replies are listed 'Best First'.
Re: howto parse (or determining end) of a line of perl by BrowserUk (Patriarch) on Aug 25, 2016 at 01:15 UTC
My solution for my REPL is very simple; I use two consecutive semicolons to indicate the end of input; and I don't evaluate the input until I see it. I just accumulate input lines, removing any newlines until I see my ad-hoc end-of-input indicator and then evaluate the entire expression. It is far simpler than trying to determine whether what you've got so far constitutes a complete expression. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]
Re^2: howto parse (or determining end) of a line of perl by perl-diddler (Chaplain) on Aug 25, 2016 at 01:48 UTC
How would that be different than simply: `home> perl ... ... ^d` [download] I.e. perl reads from stdin until you press the end-of-file char (on *nix, ^d), then it would eval everything at once. Remember, by default, I'm just doing 1 line at a time (i.e. "calculator mode")...	[reply] [d/l]
Re^3: howto parse (or determining end) of a line of perl by BrowserUk (Patriarch) on Aug 25, 2016 at 03:05 UTC
Maybe this clarifies things? `C:\test>p1 [0]{} Perl> print 123 * 346;; 42558 [0]{} Perl> print 123 * 456; print 'pqr';; 56088 pqr [0]{} Perl> sub xyz{ my( $a, $b ) = @_; my $c = $a + $b; $c += $a * $b; $c += $a - $b; $c += $a / $b; return $c ** 2; };; [0]{} Perl> print xyz( 123, 456 );; 3173549946.78328 [0]{} Perl> Terminating on signal SIGINT(2) C:\test>` [download] With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l]
Re^4: howto parse (or determining end) of a line of perl by perl-diddler (Chaplain) on Aug 25, 2016 at 18:12 UTC
Re^5: howto parse (or determining end) of a line of perl by BrowserUk (Patriarch) on Aug 25, 2016 at 18:26 UTC
Some notes below your chosen depth have not been shown here
Re^2: howto parse (or determining end) of a line of perl by Discipulus (Canon) on Aug 25, 2016 at 07:43 UTC
is BrowserUk's `REPL` available somewhere? Let's say for testing... ;=) L* There are no rules, there are no thumbs.. Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.	[reply] [d/l] [select]
Re^3: howto parse (or determining end) of a line of perl by BrowserUk (Patriarch) on Aug 25, 2016 at 12:45 UTC
No. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]
Re^3: howto parse (or determining end) of a line of perl by Anonymous Monk on Aug 29, 2016 at 14:48 UTC
If you search CPAN for REPL and you will find many REPLs that people are willing to share with you.	[reply]
Re: howto parse (or determining end) of a line of perl by Anonymous Monk on Aug 25, 2016 at 00:35 UTC
Why bother? one line: ...code...; more than one: ...code...;; But, to parse perl use PPI/PPIx::XPath ( example Perl::Critic::Policy::ValuesAndExpressions:: GivenWhenTryCatchLexicalDollarUnderDefaultVarBindingConfusion ) `$ ppi_dumper 2 PPI::Document PPI::Statement::Compound PPI::Token::Word 'for' PPI::Structure::List ( ... ) PPI::Statement PPI::Token::Number '1' PPI::Token::Operator '..' PPI::Token::Number '10' PPI::Structure::Block { ... ??? $ ppi_dumper 3 PPI::Document PPI::Statement::Compound PPI::Token::Word 'for' PPI::Structure::List ( ... ) PPI::Statement PPI::Token::Number '1' PPI::Token::Operator '..' PPI::Token::Number '10' PPI::Statement::Null PPI::Token::Structure ';' PPI::Token::Whitespace '\n'` [download] So a PPI::Statement::Compound without a closing block parenthesis, means continue `$ cat 2 for(1..10){ $ cat 3 for(1..10);` [download]	[reply] [d/l] [select]
Re^2: howto parse (or determining end) of a line of perl by perl-diddler (Chaplain) on Aug 25, 2016 at 00:58 UTC
Yikes! Yeah, I could do that... did you catch the part about not rewriting a parser? sigh... Why do it? I want to be able to enter input and see the result as I can in bash (as an example). Sure I can exec code, but then I could print the vars from vars that received an assignment in another statement (all of the user-input code in my calc is done in a separate namespace, so vars in that namespace can be preserved. PPI might allow me to do what I want, but it certainly doesn't look like a simple upgrade, but more like a redesign or such... thanks for the pointer though, might be usable... BTW as far as why I would want to display results after entering a line: remember the prog is still a calculator, like: `> pcalc pcalc V0.1.8: Type 'constants' to see constants (1)> constants Constants: Phi,Φ,phi,ɸ,pi,π,e,c,g = 1 (2)> sub area($){ my $radius=shift; 4$radiuspi } (3)> area 1 = 12.566370614359173` [download]	[reply] [d/l]
Re^3: howto parse (or determining end) of a line of perl by Anonymous Monk on Aug 25, 2016 at 01:22 UTC
Yikes! Exactly!	[reply]
Re: howto parse (or determining end) of a line of perl by choroba (Cardinal) on Aug 25, 2016 at 07:36 UTC
How have you implemented the calculator? If you already have a parser, just check whether it produces the starting symbol, if not, ask for more input. See for example my Marpa::R2 calculator with variables tolerant to missing semicolons on GitHub. ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l]
Re: howto parse (or determining end) of a line of perl by Eily (Monsignor) on Aug 25, 2016 at 09:23 UTC
I think I remember LanX talking about a REPL of his where a new line would be prompted and concatenated to the current statement until it did compile. To check for compilation without actually executing anything you can do something like: `eval "sub DUMMY { $code }; 1" or die "Failed! $@"`. Any use statment or BEGIN block will still be executed though. Of course this requires a way to cancel the current input, maybe with an EOF token in Linux, or anything that you are not going to put in actual perl code. Edit: rephrased the first sentence for clarity.	[reply] [d/l]
Re^2: howto parse (or determining end) of a line of perl by perl-diddler (Chaplain) on Aug 26, 2016 at 00:28 UTC
`eval "sub DUMMY { $code }; 1" ..."` -- that does something if "$code" contains a sub, but if you are willing to allow for "less than perfect" (fine for most personal toys), then yeah... don't really need the sub on the outside as it doesn't really provide any extra protection that I know of. Another layer of "less than perfect" protection, but w/the advantage of getting a 'default' (overridable) namespace protection, is to put the evaluated code in a separate package namespace -- so all my calculator's namespace evals are done in a 'USER' namespace -- so user routines and constants default to be separated from my machinery namespace (the keyword in many of these solutions being 'default' to allow for simple, DWIM behavior). I think there might be some better sandboxing mods in CPAN, if you wanted to get paranoid, but might not be worth the overhead/side effects for a personal 'toy'....;-)	[reply] [d/l]
Re^3: howto parse (or determining end) of a line of perl by RonW (Parson) on Aug 26, 2016 at 19:19 UTC
don't really need the sub on the outside as it doesn't really provide any extra protection that I know of It prevents the code from being executed. Except, of course, for BEGIN blocks and use statements (which also execute when you use the -c (compile only) option to perl).	[reply]
Re: howto parse (or determining end) of a line of perl by Anonymous Monk on Aug 25, 2016 at 00:42 UTC
Good luck... You could try eval on what you have and if it gives an error change prompt and read more :) That may work some of the time, especially if you leave an operator at the end of the line. (I have a much better way, but the margins of this note don't have enough room to explain it.)	[reply]
Re^2: howto parse (or determining end) of a line of perl by perl-diddler (Chaplain) on Aug 25, 2016 at 00:47 UTC
Yeah thought about that (eval+error->reprompt). Problem w/that is that evaluating it could change the 'environment' such that re-evaluating it later wouldn't work. (Example in my margins; ;-))	[reply]
Re^3: howto parse (or determining end) of a line of perl by RonW (Parson) on Aug 26, 2016 at 19:30 UTC
Maybe starting a thread to test compile the expression. "Return" success or fail to the main then end the thread. (Maybe I'm misremembering, but I think that threads in Perl work by creating additional PVMs to execute the threads.) Or try using Safe (which is a core module) to test-compile the expressions.	[reply]