Hello monks,

I've mostly written a Perl program that runs several external backup and file transfer programs, such as duplicity, scp, rsync, etc. Everything's fine except I'm not sure of the best way to actually execute the commands. I'll show you the relevant subroutine, then I'll get into specifics:

# Run executable @system command, reporting name as $name If there 
# is an upload limit, we run through trickle(1) to limit bandwidth.
sub _ext_cmd {
    my ($name, @system) = @_;
    
    if ($o{general}{upload}) {
        @system = ($o{general}{trickle},
                   $o{general}{trickled} ? () : '-s',
             -u => $o{general}{upload}, '|', @system);
    }        
    say "Running name=$name, @system";
    # system { $system[0] } @system[1..$#system]; # Obviously nope.
    say "\$?=$?";
}

[download]
# Excerpt of %o options hash:
%o = (
    general => {
        upload   => '256', # KBps
        trickle  => '/bin/trickle',
        trickled => undef, # True if trickled is installed
    },
);

# Example usage:
_ext_cmd(display_name => qw!/bin/duplicity /path/to/src /path/to/dest!
+);
# ... but you could probably test it just fine with echo or /bin/cat.

[download]

A couple of important points:

  1. The commands will often be in a pipeline with the bandwidth-limiting utility trickle(1)
  2. In general, the commands are long-running (hours, even days for first backups) but not indefinite. Several megabytes of output are typically generated.
  3. stdout and stderr need to be logged to file (captured might work, although I'll be keeping the logs for later browsing, and capturing gains me nothing except wasted RAM). Exit codes are important, too.
  4. As usual, I'd rather not have to escape spaces and the like... thus passing arguments in an array is preferred.
  5. CPAN modules are fine, if necessary.

Right, that was more than a couple points. I swear I've done this a hundred times before, but for whatever reason my distracted brain doesn't want to put this puzzle together on its own today.

In reply to Capturing output from a long-running pipeline by wanna_code_perl

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.