symgryph has asked for the wisdom of the Perl Monks concerning the following question: 

#!/usr/bin/perl -w
#this file is for grepping within vdx files, and outputs the following
+:
#filename:<#of hits>:<arguments>
#filename:2:myargument
open ARGUMENTS, "<","file";
@files = <*.vdx>;
@arguments = <ARGUMENTS>;
foreach $file (@files) {
    foreach $argument (@arguments) {
    chomp $argument;
    `grep -c -H $argument \"$file\"`;
    chomp $results;
    print "$results:$arguments\n";
    }
}

[download]

I am writing the above code whoose purpose is to recursively go through a list of files, with a list of parameters to search for, and then print things that match, as well as their matching string.

The problem is that many of the filenames contain spaces etc. that grep hates and interprets as input. I figured out that grep can use quotes in order to 'escape' the spaces when searching through files, and using shell this works fine, but when execcing via the backtick operator, nothing happens, as if grep isn't being sent any filename.

Below follows with the two files, arguments, (aka file in the program), and the 'glob' pattern that matches all things it the current directory named 'vxd'. The script is then supposed to execute grep with quotes in the filename and print something like: filename:6:things_that_matched The '-H and the -c" in grep cause grep to print the count of things that matched, and -H prints the filename. I couldn't figure out how to get grep to print out the original argument that it was searching for. 

hello
goodbye

[download]
"Two Wheels good, Four wheels bad."

Replies are listed 'Best First'.
Re: Perl backticks and GREP?
by moritz (Cardinal) on Oct 19, 2009 at 16:36 UTC
    A good solution is to open the files with perl, using the three-argument form of open (which correctly handles whitespaces in filenames) and use Perl regexes to search for the patterns.

    You should also Use strict and warnings, and declare your variables - it catches at least two errors in your script.

    Perl 6 - links to (nearly) everything that is Perl 6.
[reply]
Re: Perl backticks and GREP?
by toolic (Bishop) on Oct 19, 2009 at 16:38 UTC
    `grep -c -H $argument \"$file\"`;
    I think you want to capture the grep output into a variable: 
        $results = `grep -c -H $argument \"$file\"`;

    [download]
    See qx.
[reply]
[d/l]
Re: Perl backticks and GREP?
by jakobi (Pilgrim) on Oct 19, 2009 at 17:12 UTC

    See also my recent Re: *SAFE* use of string in system command for safe use of filenames.

    Use %ENV and the old standby of adding /dev/null to force grep to print filename**s**. That way names containing quotes and worse won't create a mess.

    # note that a simple pure perl version using 3arg open and
# e.g. $count=$contents=~s/$argument/$&/g will have 
# about the same code length; besides offering way more
# **cute** regexes... 

$ENV{file}=$file;         # protect filename from shell interpolation
$ENV{argument}=$argument; # also protect non-trivial regexes from shel
+l!

# upd: either -H OR /dev/null, sigh. a very telling misread of mine
$results=`grep -c -H "\$argument" "\$file"`; # /dev/null`; # note \$ i
+nstead of \"

    [download]

    Above is as usual incomplete and a bold lie: the reporting with still suffer in case of an embedded but unfiltered \n being a part of the filename (UTF-8 probably also offers some more interesting whitespace to play with; names with terminal esc sequences are also fun in print, warn & die). But especially \n really warants IMAO a cronjob and a shoot-and-kill policy for both file and file creator upto and including layer 8.

    cu
    Peter

    Update:

    1. of course, this (unnecessary) last minute addition of /dev/null belongs before the closing `-quote, just as your first parsing error indicated. Sorry & fixed above.

    2. Oops: you need to move down the $ENV{file} (in your example below) into the first foreach loop. The $ENV{argument} similar after $argument has been assigned to, into the second loop, or better yet, trading efficiency to readability, both just before the grep.

    3. ok, compare your version with the full working one here and do check the comments:

    Read more... (2 kB)
[reply]
[d/l]
[select]
      #!/usr/bin/perl -w
#this file is for grepping within vdx files, and outputs the following
+:
#filename:<#of hits>:<arguments>
#filename:2:myargument
open ARGUMENTS, "<","file";
@files = <*.vdx>;
@arguments = <ARGUMENTS>;
$ENV{file}=$file;
$ENV{argument}=$argument;
foreach $file (@files) {
    #$copyfile="\"".$file."\"";
   foreach $argument (@arguments) {
    chomp $argument;
    #print "$copyfile\n";
    $results=`grep -c -H "\$argument" "\$file"` /dev/null;
    chomp $results;
    print "$results:$arguments\n";
    }
}

      [download]

      results in the following output:

      Unquoted string "dev" may clash with future reserved word at ./nelson.
+pl line 15.
Unquoted string "null" may clash with future reserved word at ./nelson
+.pl line 15.
Use of uninitialized value in scalar assignment at ./nelson.pl line 8,
+ <ARGUMENTS> line 1.
Use of uninitialized value in scalar assignment at ./nelson.pl line 9,
+ <ARGUMENTS> line 1.
grep: : No such file or directory
Argument "dev" isn't numeric in division (/) at ./nelson.pl line 15, <
+ARGUMENTS> line 1.
Argument "" isn't numeric in division (/) at ./nelson.pl line 15, <ARG
+UMENTS> line 1.
Illegal division by zero at ./nelson.pl line 15, <ARGUMENTS> line 1.

      [download]
      "Two Wheels good, Four wheels bad."
[reply]
[d/l]
[select]
        Unquoted string "dev" may clash with future reserved word at ./nelson.
+pl line 15.
Unquoted string "null" may clash with future reserved word at ./nelson
+.pl line 15.
...
Argument "dev" isn't numeric in division (/) at ./nelson.pl line 15, <
+ARGUMENTS> line 1.
Argument "" isn't numeric in division (/) at ./nelson.pl line 15, <ARG
+UMENTS> line 1.
Illegal division by zero at ./nelson.pl line 15, <ARGUMENTS> line 1.

        [download]

        All of the above are generated by the /dev/null you have marooned outside the backticks. Are you attempting to redirect the output of grep?

        Use of uninitialized value in scalar assignment at ./nelson.pl line 8,
+ <ARGUMENTS> line 1.
Use of uninitialized value in scalar assignment at ./nelson.pl line 9,
+ <ARGUMENTS> line 1.

        [download]
        You don't seem to have initialised $file or $argument anywhere before trying to use them.

        grep: : No such file or directory

        I don't think you should be escaping the variables inside the backticks. They need to interpolate.

        I hope these points help you move forward.

        Cheers,

        JohnGG

[reply]
[d/l]
[select]
Re: Perl backticks and GREP?
by leocharre (Priest) on Oct 19, 2009 at 16:38 UTC
[reply]

Back to Seekers of Perl Wisdom