Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have a script that generates several fairly complicated SVGs, places them in a folder, and then runs an external Java program to convert the folder of SVGs into a single PDF. Currently I'm calling the Java program from within my script as follows:

$report_failure = system('java -jar /var/www/helper_code/svg2pdf.jar ' + . "$output_files$invoice_data_ref->{'actual_invoice_id'}" . '/master +*.svg' . ' ' . "$output_files$invoice_data_ref->{'actual_invoice_id'} +" . '/master.pdf');
This works, but it is slower than I expect it to be. For example, if I generate the folder of SVGs, and then manually run the Java program from the shell it takes 5-6 seconds to create the PDF. However, executing the Java command from within my perl script takes 13 seconds. At first I thought this was just due to the overhead of calling any external command from within perl, but if I replace my call to system with a simpler call (e.g., system('java -version') ) then that call takes less than a second.

Any ideas on how to figure this out?

Replies are listed 'Best First'.
Re: Speeding up external command execution
by BrowserUk (Patriarch) on Apr 13, 2014 at 22:48 UTC
    Any ideas on how to figure this out?

    Yes. Use top or tasklist to monitor both the perl executable and the java executable during the command execution; for cpu, IO & memory usage.

    In theory, the Perl instance should show negligible activity in all three categories. If that is not the case, post your findings and someone here may be able to shed some light on the possibilities.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Speeding up external command execution
by graff (Chancellor) on Apr 14, 2014 at 00:24 UTC
    While RAM limitations are rarely a concern these days, it's about the only thing I can imagine that might cause the timing difference (apart from possible environment differences hinted at by soonix above) - i.e. the java process takes 5-6 seconds, given the available RAM when it's running alone, but if your perl script is running at the same time (and if it takes up a lot of RAM), then you might lose a lot of time to memory swapping (aka page faults). BrowserUK's suggestion will show you if that's the issue.

    Apart from the timing issues, the logic you're using to build the command line is strange. I would have done it something like this:

    my $pathstr = join( '', $outputfiles, $invoice_data_ref{actual_invoice +_id} ); my @cmdargs = ( 'java', '-jar', '/var/ww/helper_code/svg2pdf.jar', "$pathstr/master*.svg", "$pathstr/master.pdf" ); print join( ' ', 'command_str:', @cmdargs ), "\n"; # optional - just +to check $report_failure = system( @cmdargs );
    My version should yield the same behavior as yours, but if, instead of using "master*.svg", you were to use the actual list of file names (that is, the names of the svg files that were just created previously in your script), like this:
    my @svg_names = ( qw/master01 master02 .../; # or whatever those names + are # … create those files ... my $pathstr = join( '', $outputfiles, $invoice_data_ref{actual_invoice +_id} ); my @cmdargs = qw/java -jar /var/ww/helper_code/svg2pdf.jar/; push @cmdargs, "$pathstr/$_.svg" for ( @svg_names ); push @cmdargs, "$pathstr/master.pdf";
    then the method of execution will be slightly different. If you haven't read perldoc -f system, you might find that interesting.

    I expect that after you generate all those "fairly complicated SVGs", you close those file handles (i.e. make sure all the output has been flushed) before doing the system call. If those file handles are still open, perl will flush them for you when you do the system call, which might add a bit of delay.

    Your description suggests that the system call is only being done once; if you were doing a lot of similar system calls in some sort of loop, then one possible cause of delay would be the extra overhead of invoking and cleaning up after a subshell on each iteration; one way to get around that (on a *n*x system) would be:

    open( SH, '|-', '/bin/sh' ) or die "Can't open a shell: $!"; for my $iterator ( @whatever_set ) { print SH "command line…\n" } close SH;
    But that's probably not relevant here.
Re: Speeding up external command execution
by eye (Chaplain) on Apr 13, 2014 at 21:53 UTC
    Have you tried taking the command you ran from the shell and wrapping it in a minimal Perl script that is just a system() call?
    #!/usr/bin/perl use strict; use warnings; system( '...' ); exit;
Re: Speeding up external command execution
by frozenwithjoy (Priest) on Apr 13, 2014 at 19:35 UTC
    What happens if you call it from within perl without looking up values (i.e., try it hard-coded)? This would at least answer the question of whether the problem arises from calling it in perl or from some artifact related to the way you are putting the call together. Also, are you sure that it is solely this call that is slowing things down and not other stuff going on before/after?
      If I hard code the command it still takes 13 seconds. And I'm sure the delay is coming from just this call because I'm timing it by calling time() just before and just after the call to system.
Re: Speeding up external command execution
by boftx (Deacon) on Apr 13, 2014 at 19:44 UTC

    EDIT: Kindly disregard this post as it has become moot in view of the reply to the preceding post that in fact answers my question.

    This might be a dumb question, but how are you determining the execution times? If you are using the obvious method of time command on the command line then you are including the startup time for perl, which may be significant depending upon hardware, OS, etc.

    It helps to remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.
Re: Speeding up external command execution
by soonix (Chancellor) on Apr 13, 2014 at 21:33 UTC
    Does system('env') differ from the shell's env?
Re: Speeding up external command execution
by basiliscos (Pilgrim) on Apr 14, 2014 at 08:08 UTC

    Java is notorious for it's slow startup. If you execute java hepler for some web-applications from user requests, it could be performance killer for your application (e.g. as I think, if you run 10 java instances in simultaneously)

    So, if you have access to java-sources and know java a little bit, to rewrite your helper as network daemon, listening local interface and communicate with it via sockets. At least problem with "startup overhead", which you attribute to perl, will disappear.