idlehands has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to use backticks to run ls to search for files to gather a list of files that exist. End goal is to parse a group of iTune's playlist files to make sure when import the playlists to itunes it won't drop files if they don't exist. My problem is when I run ls in backticks with the file name, a \r is post pended to the command. making every file show up as not found.

$playListRow = quotemeta($playListRow); my $lscom = "ls $playListRow 2>&1"; my $result = `$lscom`;

this outputs: ls: /Volumes/public_files/music/ABC - The Lexicon of Love (1982) {1996 Remaster} V0what.cd/16 Look Of Love (Live).mp3\r: No such file or directory

No matter how I fiddle with it I can't get the \r to go away. Thanks!

Replies are listed 'Best First'.
Re: backtick iterpolation issue
by toolic (Bishop) on Jul 01, 2015 at 13:05 UTC
    I can't get the \r to go away.
    The substitution operator will remove all \r from a string:
    use warnings; use strict; my $result = "/Volumes/public_files/music/ABC - The Lexicon of Love (1 +982) {1996 Remaster} [V0]what.cd/16 Look Of Love (Live).mp3\r"; $result =~ s/\r//g; print ">>>$result<<<\n"; __END__ >>>/Volumes/public_files/music/ABC - The Lexicon of Love (1982) {1996 +Remaster} [V0]what.cd/16 Look Of Love (Live).mp3<<<
Re: backtick interpolation issue
by Athanasius (Cardinal) on Jul 01, 2015 at 13:47 UTC

    Hello idlehands, and welcome to the Monastery!

    As toolic says, you can remove carriage returns with the substitution operator. The transliteration operator is another option:

    $result =~ tr/\r//d;

    But most likely you don’t need backticks at all: forget ls and just use Perl’s built-in commands. For example, to get a list of all the plain files (but not directories) in directory $dir:

    use strict; use warnings; use autodie; use Data::Dump; my $dir = ...; opendir(my $dh, $dir); my @f = grep { -f } readdir($dh); closedir $dh; dd \@f;

    See readdir and the file test operators.

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

Re: backtick iterpolation issue
by choroba (Cardinal) on Jul 01, 2015 at 13:03 UTC
    How do you populate $playListRow? Can you show it's contents through
    use Data::Dumper; $Data::Dumper::Useqq = 1; print Dumper($playListRow);
    لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
       $VAR1 = "\\/Volumes\\/public_files\\/music\\/The\\ Lexicon\\ Of\\ Love\\/4\\.Tears\\ Are\\ Not\\ Enough\\.mp3\\\r"; playlist row is just a line pulled from an itunes m3u file, though I filtered out the comments from the m3u file:
      open(my $fh2, '<:encoding(UTF-8)',$playlistName) or die "Could not open playlist file '$playlistName $! +"; while( my $playListRow = <$fh2>){ chomp $playListRow; if($playListRow =~ /^\s*#/){ next;} else{ # print "$playListRow\n"; $playListRow = quotemeta($playListRow); my $lscom = "ls $playListRow 2>&1";
        Replace chomp with
        $playListRow =~ tr/\xd\xa//d;

        It should remove all the newlines, CR/LF or LF style.
        Update: You might need to verify how file names containing newlines are represented in the list.

        لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
Re: backtick iterpolation issue
by dave_the_m (Monsignor) on Jul 01, 2015 at 13:42 UTC
    quotemeta won't do what you want it to. It is intended mainly for inserting fixed strings into regular expressions. It's definitely not suitable for escaping shell arguments; in particular, it doesn't escape spaces. (update: that's a lie; it *does* quote spaces).

    You probably want to use a glob instead of forking out to ls, e.g.

    my @files = <$playListRow/*>

    Dave.

Re: backtick iterpolation issue
by 1nickt (Canon) on Jul 01, 2015 at 13:51 UTC

    If there's no good reason not to, you should use a CPAN module for a task like this. Someone else has already done all the work!! Here's one way:

    #!/usr/bin/env perl use strict; use warnings; use 5.010; use Path::Iterator::Rule; my $file = 'ABC - The Lexicon of Love (1982) {1996 Remaster} V0what.cd +/16 Look Of Love (Live).mp3'; my @dirs = '/Volumes/public_files/music'; my $rule = Path::Iterator::Rule->new; $rule->name($file); my $search = $rule->iter( @dirs ); while ( my $found = $search->() ) { say $found; } __END__

    The good thing about using a module like this is that when you figure out some new requirement, like for example you only want to search three levels down, or the file has to be a certain minimum or max size, or has to contain a certain string etc etc, these are all built-in functions, so you can add your new functionality without hardly having to break a sweat:

    $rule->name($file) ->size("> 1000");
    Remember: Ne dederis in spiritu molere illegitimi!
Re: backtick iterpolation issue
by trippledubs (Deacon) on Jul 01, 2015 at 13:59 UTC
    You need chomp
    #!/usr/bin/env perl use strict; use warnings; my @files = `ls *ext`; my $wanted = $files[0]; print $wanted; print "Done$/$/$/"; # $/ = $INPUT_RECORD_SEPARATOR ie newline chomp($wanted); print $wanted; print quotemeta($wanted); print "Done";
    Output:
    Weird File Name with Charactesr(1982){1996 Remaster}.ext Done Weird File Name with Charactesr(1982){1996 Remaster}.extWeird\ File\ N +ame\ with\ Charactesr\(1982\)\{1996\ Remaster\}\.extDone
    You can even use in one line
    chomp(my @files = `ls`);
Re: backtick iterpolation issue
by idlehands (Initiate) on Jul 01, 2015 at 17:01 UTC
    here's the winner:
    $playListRow = shell_quote($playListRow); $playListRow =~ s/\r//g; my $result = `ls $playListRow `; print "$result\n";
    Filtered the /r out, and had to switch to shell_quote from quoteMeta as quoteMeta wasn't working quite right. I was using backticks as I needed to get the stderr and stdout. At a high level what I'm trying to do: parse a directory of iTunes playlists(m3u files) determine whether all the files in the playlist exist on my file server. If they do great, if they don't, output a playlist file, that lists which tracks are missing, what they are and where they go in the track order. Once i have that complied, I can import my playlists one by one and know whether I have the whole playlist, or if it's missing tracks. If it's missing tracks I can fix it. The issue i've been fighting with is if you import a playlist to itunes and are missing some of the files, itunes just dumps them. No error, nor any log that I've found. Thanks for all the help!
      Fine if it works and provides what you want. But I would still reiterate the advice provided earlier by other monks: there is really no reason to shell out and run an ls shell command when Perl provides you with powerful builtins to do just that, such a glob (my favorite choice in this specific case) or opendir/readdir.

      Update: added links to the documentation of the suggested functions.

      I was using backticks as I needed to get the stderr and stdout.

      But using backticks as you did doesn't capture the stderr of your system executable!

      #!/usr/bin/env perl -w use strict; my $foo = qx{ ls }; # alternate syntax for backticks open my $fh, '>', 'bar.out' or die $!; print $fh $foo; close $fh; __END__
      $ cat bar.out foo.pl $
      #!/usr/bin/env perl -w use strict; my $foo = qx{ ls baz }; # doesn't exist open my $fh, '>', 'bar.out' or die $!; print $fh $foo; close $fh; __END__
      $ cat bar.out $

      You may have seen an error message spit out to your screen by your system executable, but it was not captured in your Perl program because you used backticks. You can combine the output streams, depending on your OS, with something like:

      #!/usr/bin/env perl -w use strict; my $foo = qx{ ls baz 2>&1 }; open my $fh, '>', 'bar.out' or die $!; print $fh $foo; close $fh; __END__
      $ cat bar.out ls: baz: No such file or directory $

      But then you'll have to parse what you get to see if matches an error string. This all begs the question of why you didn't follow advice from others as well as myself to use a module that is built for the task ... because, among other benefits they usually handle errors gracefully.

      Remember: Ne dederis in spiritu molere illegitimi!