Re: Extending a perl program with Scheme, Lua, or JS
by haukex (Archbishop) on Feb 08, 2019 at 07:02 UTC
|
use warnings;
use strict;
use JE;
my $j = new JE;
$j->eval(<<'MYJSCODE');
function foo (x) {
return x / 10;
}
MYJSCODE
my $rv = 0+$j->{foo}->(50);
print "$rv\n"; # prints "5"
| [reply] [d/l] |
|
|
| [reply] |
|
|
Hm, the CPAN page for JE has this: "It also uses and leaks lots of memory."
Yeah, good point, I missed that initially, it's definitely something worth testing. I would hope that if the JE object gets destroyed, it cleans up all of its memory, so that might be a simple workaround.
Startup time for the interpreter is 85 ms, which is not great
Initially you said "in my actual use case the performance hit might be undetectable to the person (me) using the GUI" - what kind of performance are you looking for here? Is this code going to be called in a tight loop, how often, etc.?
| [reply] [d/l] |
|
|
|
|
|
Re: Extending a perl program with Scheme, Lua, or JS
by choroba (Cardinal) on Feb 07, 2019 at 23:10 UTC
|
For simple arithmetic operations, you can use Marpa::R2. A simple calculator is shown directly in the Synopsis. I'm not sure whether it exists as a debian package, but I guess it does.
Also, please linkify the link in your post. [id://537912] → Is there a way to call lisp from perl?.
map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
| [reply] [d/l] [select] |
|
|
Thanks, Marpa looks nice. I couldn't get the sample program to run, though:
Can't locate object method "parse" via package "Marpa::R2::Scanless::G" at b.pl line 24.
I guess the other disadvantage is that I would then have to design and implement my own language using Marpa. This could be simple for very simple arithmetic operations, but inevitably I can imagine then having to extend it. Probably better to start with someone else's good design and implementation.
| [reply] [d/l] |
|
|
Maybe your version of Marpa is too old? Consult
perldoc Marpa::R2
locally instead of the online manual of the latest version.
map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
| [reply] [d/l] [select] |
Re: Extending a perl program with Scheme, Lua, or JS
by Laurent_R (Canon) on Feb 08, 2019 at 00:02 UTC
|
Not entirely sure to understand what you really need and why you can't just perform the arithmetic operations with Perl. If it boils down to evaluating arithmetic expression, maybe you could use eval, but there might be some security problems, so you have to check that the expression to be evaluated is safe prior to doing it.
Another possible option that comes to my mind is to shell out to the bc Unix or Linux utility. bc can be used in non-interactive mode in various ways, including piping (echo "42/7" | bc), shell redirections, and Un*x heredocs.
| [reply] [d/l] [select] |
|
|
Another possible option that comes to my mind is to shell out to the bc Unix or Linux utility. bc can be used in non-interactive mode in various ways, including piping (echo "42/7" | bc), shell redirections, and Un*x heredocs.
How to make things even worse. I smell shell injection: "Dear script, please evaluate 1 + $(rm -rf /) or 1 + ";rm -rf /;echo " for me." (echo "1 + $(rm -rf /)" | bc resp. echo "1 + ";rm -rf /;echo "" | bc) Shelling out ain't that easy if you want to do it right: The problem of "the" default shell
And even if you shell out safely, the program you feed the input into may have security issues. See Morris_worm for a really old example, or CVE-2012-1165 for a younger one.
Alexander
--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
| [reply] [d/l] [select] |
|
|
Yes, you're absolutely right, the type of security problems I mentioned about eval also apply to shelling out, I should have mentioned that explicitly. I actually don't like much the idea of shelling out, my point in that paragraph was only to say that if one decided to shell out, it might be simpler to use bc rather than heavy artillery such as an interpreter, compiler, or VM for Lisp, Lua, or JS.
| [reply] [d/l] [select] |
|
|
Thanks for the warning, but as mentioned in the OP, it's not a web app.
| [reply] |
|
|
|
|
|
|
|
|
|
Not entirely sure to understand what you really need and why you can't just perform the arithmetic operations with Perl
If other people are going to use it, I don't really want them to have to learn perl syntax. It looks messy and confusing to the uninitiated, with all the sigils, and the nonstandard mechanism for passing arguments to functions. I don't know, maybe it's just my perception, but say you show someone who has some basic coding experience this code:
sub f {$x=shift; return $x*0.1}
If they've never seen perl before, they're going to wonder what the heck the dollar signs are, and what shift is. The equivalent in Scheme is this:
(define (f x) (* x 0.1))
Here, I think the only possibly mysterious thing would be that you have to realize that it's prefix rather than infix notation. Maybe for zero-surprises pure vanilla syntax, JS wins:
function f(x) {return x*0.1}
However, the only JS interpreter that seems to meet my criteria is Rhino, and it seems to have unacceptable performance for my purposes if I try to start up an interpreter once for each operation:
$ time rhino -e 'function f(x) {return x*0.1}; print(f(34))'
3.4000000000000004
real 0m0.240s
user 0m0.292s
sys 0m0.036s
(Rhino is actually amazingly fast once it starts up -- in my experience it's sometimes faster than perl, and within a factor of 2 or 3 compared to C.)
Another possible option that comes to my mind is to shell out to the bc Unix or Linux utility. bc can be used in non-interactive mode in various ways, including piping (echo "42/7" | bc), shell redirections, and Un*x heredocs.
Nice suggestion, thanks. It's definitely nice in terms of performance and zero-surprises syntax:
time echo "define f(x) {x*0.1}; print f(34)" | bc
3.4
0
real 0m0.002s
user 0m0.000s
sys 0m0.000s
I think the only problem from my point of view might be that I will probably sometimes need to use data structures, which bc doesn't have. E.g., a common application would be that you want to drop the student's lowest quiz score, so you need to pass the list of quiz scores to the function. (Another issue would be that this would make it linux-only.)
| [reply] [d/l] [select] |
|
|
After originally implementing this in Guile, I've ended up switching to bc, because I couldn't figure out any clean way of preventing the possiblity of Trojan horse attacks if I was using Guile.
| [reply] |
Re: Extending a perl program with Scheme, Lua, or JS
by bliako (Abbot) on Feb 10, 2019 at 14:14 UTC
|
Indeed (re: Re^2: Extending a perl program with Scheme, Lua, or JS)! That's exactly how I got my degree, got out of a lengthy prison sentence and keep adding extra money to my bank account every month. All via a script.
Seriously now, you may believe that defining a function (in X language) to take in data structures (plain arrays which can soon become Perl's objects or hashes) and then, I guess, modify these data structures back in the main Perl program is trivial. And all these for the sake of avoiding a shift. But you may be happier if you reconsider. For example shift can also be written as $_[0] or using the power of OO:
my ($teacher_obj, $student_obj, $new_grades) = @_;
$student_obj->teacher_modify_grades($new_grades, $teacher_obj);
The argument for keeping within Perl can only be strengthen by looking at modules like dc, bc which are not shelling out to bc but implement its operation in Perl or C. And of course, you can create any calculator to suit your needs using Marpa by following MarpaX::ESLIF::Tutorial::Calculator.
In any event, I would keep the calculator for calculating. And provide an API for the higher-level operations. But that's me.
| [reply] [d/l] [select] |
Re: Extending a perl program with Scheme, Lua, or JS
by karlgoethebier (Abbot) on Feb 13, 2019 at 08:17 UTC
|
Math::RPN?
«The Crux of the Biscuit is the Apostrophe»
perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help
| [reply] [d/l] |
Re: Extending a perl program with Scheme, Lua, or JS
by karlgoethebier (Abbot) on Feb 13, 2019 at 18:40 UTC
|
He! CPAN is magic 😎 See also Tk::Calculator::RPN::HP. I don‘t know if this module is good or useful for you. Regards, Karl
«The Crux of the Biscuit is the Apostrophe»
perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help
| [reply] [d/l] |
Re: Extending a perl program with Scheme, Lua, or JS
by bcrowell2 (Friar) on Feb 11, 2019 at 22:33 UTC
|
Thanks, all, for the helpful advice and pointers!
I've ended up deciding to go with Guile via a shell, at least for the time being. Here is my glue code. Guile meets my criteria of being installable as a debian package, having good performance, being stable and actively maintained, supporting data structures, and having simple syntax.
Although I think the possibility of a Trojan horse attack using Guile code embedded in a document is realistically a little remote due to my small user base plus sociological factors, I'm looking into how to sandbox Guile. Guile has a sandbox mechanism that seems to be only focused on preventing excessive use of resources. I've posted on the guile-user email list to ask if there's any way to prevent code from accessing the file system and such. I could also just turn off Guile extensions by default. | [reply] |
|
|
Here is my glue code.
use strict;
package Extension;
sub apply_scalar_function_of_x {
my $f = shift; # lisp source code, with x as the input, e.g., "* x 0
+.1"
# As a convenience feature, if $f is undefined or nul
+l string, we return the input unaltered.
my $x = shift;
if (!defined $f) {return $x}
my $lisp = "(display ((lambda (x) ($f)) $x))";
my $result = `guile -c '$lisp'`;
if ($? == 0) {
return $result;
}
else {
print STDERR "Error executing scheme code $lisp using guile.";
return undef; # occurs if guile is not installed or the guile code
+ dies with an error
}
# to do:
# Escape single quotes inside the string.
}
1;
Unfortunately, as discussed, this suffers from multiple potential injection and quoting issues. In this case, I would recommend either IPC::Run3 (libipc-run3-perl) or IPC::System::Simple (libipc-system-simple-perl) - the former is probably better if this software is supposed to run on Windows too. If you don't want to install an extra module, then you could use the piped open I show near the bottom of my post here.
use IPC::System::Simple qw/capturex/;
sub apply_scalar_function_of_x {
my $f = shift;
my $x = shift;
if (!defined $f) {return $x}
my $lisp = "(display ((lambda (x) ($f)) $x))";
my $result;
if (not eval { $result = capturex('guile','-c',$lisp); 1 }) {
print STDERR "Error executing scheme code using guile.\n";
return;
}
return $result;
}
I'm looking into how to sandbox Guile. ... if there's any way to prevent code from accessing the file system and such.
Yes, I would very strongly recommend not leaving an open source grading system open to attacks ;-)
| [reply] [d/l] [select] |
|
|
Thanks, I've redone the code as you suggested:
https://github.com/bcrowell/opengrade/blob/master/Extension.pm
However, Scheme is a general-purpose programming language, and as I've pointed out previously, this feature is designed to run user-defined scheme code. So there's no need to do injection or quoting. If you're a malicious person writing a Trojan horse, the thing to do would be simply to put the malicious code in there, and if you can get someone to open your file, it will run. Nothing else can fix this unless Guile can be sandboxed. Protecting against injection and quoting would become helpful only if there was also sandboxing. I've asked in a couple of places for suggestions on how to sandbox Guile:
http://lists.gnu.org/archive/html/guile-user/2019-02/threads.html#00025
https://stackoverflow.com/questions/54640307/sandboxing-guile-by-deleting-unwanted-libraries
No replies yet. It may just be necessary to turn off the extension mechanism by default, and let users know that if they activate it, they will be running arbitrary code from any person who gives them a file to open.
| [reply] |
|
|
my $result = `guile -c '$lisp'`;
I would put the full path to guile there..., or compile a guile without system()+friends support, make it static and you have some kind of sandbox. The latter is theoretical and requires better skills than mine because it seems guile's scm_system() is required for its own compilation...
| [reply] [d/l] [select] |
|
|
Thanks for the suggestions. Unfortunately I don't think this is practical. The path for guile will be different on different systems, so I can't just hardcode it. If someone has the ability to get on my desktop system and put a malicious binary named, say, ls in my home directory, I'm kind of screwed no matter what.
I also don't want to get into the business of maintaining and distributing my own sandboxed version of guile, nor do I want potential users to have to install such a thing just so they can use my code; one of my design criteria is that the program should not have any dependencies that are not available as debian packages. If I had the relevant time and knowledge of the Guile codebase, I think the thing to do would just be to offer the Guile folks a patch with an option like -s to run it sandboxed, and get that patch accepted into the standard version of Guile. But I think that would be jumping the gun, since I don't even know yet whether there is some easy, standard way to sandbox Guile.
| [reply] |