Receiving Standard Input/Piped

thekestrel has asked for the wisdom of the Perl Monks concerning the following question:

Monks
This has got to be in the silly easy category, but I can't think of it for the life of me and I can't get the right keywords in super search to show what I'm missing...
I'm wanting to process a series of files but rather than go...

-->process file1 file2 file3
[download]

...I want to be able to go some thing like

-->ls file* | process
[download]

I'm running on linux and was under the impression that if I piped data it would appear as command line arguments, but this does not seem to be the case, or at least I'm doing it wrong.
So writing a simple example(counting lines) the does the first case does not work for the second...

#!/usr/bin/perl

use strict;
use warnings;

my $numargs = $#ARGV + 1;

if ( $numargs == 0 ) {
        print "Incorrect number or arguments($numargs)...\nFormat: lin
+es <file>\n\n";
        exit(0);
}
my $t = 0;
for my $f ( 0 .. $#ARGV ) {
        my $file = $ARGV[$f];
        open FILE, "<$file" or die "Could not open file, $file : $!\n"
+;
        my $c = 0;
        while (<FILE>) {
                $c++;
                $t++;
        }
        close (FILE);
        print "$c\n";
}
print "----\n$t\n";
[download]

Could someone please enlighten me to my fault and offer a solution for processing piped content and command line args? I've got a feeling I might have to check for command arguments and then if there are none then open up the standard input stream to receive content?

Regards Paul.

Comment on Receiving Standard Input/Piped Select or Download Code

Replies are listed 'Best First'.
Re: Receiving Standard Input/Piped by sk (Curate) on Sep 08, 2005 at 15:42 UTC
`ls \| prog` will be treated as an input to the prog and not as a command line arg for example do `ls \| cat` you will get `ls` back (minus the color etc. if you had them turned on) you need `ls \| xargs cat` or `ls \| xargs prog`	[reply] [d/l] [select]
Re^2: Receiving Standard Input/Piped by thekestrel (Friar) on Sep 08, 2005 at 16:06 UTC
of course... =P So xargs just opens up standard input, reads data, reads a file per line as default and calls an instance of the following program with a command line argument of one of the files for each file. I was trying to think of any other programs that do this in one go and can process standard input as arguments also (inherant xargs), but couldn't come up with any. The closest thing I could think of that I use... `find -name "*.h" -exec grep "search word" {} \; -print` [download] looks like its just the same thing, once find has all the files, call and instance of grep with a single argument. Anyways thanks for your help, thats what I needed =). Regards Paul.	[reply] [d/l]
Re^3: Receiving Standard Input/Piped by sk (Curate) on Sep 08, 2005 at 16:14 UTC
So xargs just opens up standard input, reads data, reads a file per line as default and calls an instance of the following program with a command line argument of one of the files for each file. Actually `xargs` can-call/calls the program with multiple args. You can also specify the number of args using -n switch people prefer `xargs` because it avoids the `Argument list too long` problem. Also the `-exec` is not as efficient as `xargs` because it calls the prog each time it finds a file while xargs knows how big is OK and calls it accordingly! i.e. fewer calls to the prog cheers SK	[reply] [d/l] [select]
Re: Receiving Standard Input/Piped by Codon (Friar) on Sep 08, 2005 at 17:31 UTC
Warning: There is no Perl in this post! For what you are doing, you could get most of that via the commandline tools: `> ls * \| xargs wc -l` [download] Or, if you have a directory tree that you want to examine: `> find . -type f \| xargs wc -l` [download] This will even give you your total, but you may hit the limit of the size of a command line. Ivan Heffner Sr. Software Engineer, DAS Lead WhitePages.com, Inc.	[reply] [d/l] [select]
Re^2: Receiving Standard Input/Piped by thekestrel (Friar) on Sep 08, 2005 at 22:16 UTC
You are most certainly correct. =) But the process script that counted lines however was just a gimpy example to do something that used all the inputs. The processing is intensive but unrelated to my problem. I had actually forgotten about the word count binary though, thanks. Regards Paul	[reply]
Re: Receiving Standard Input/Piped by Tanktalus (Canon) on Sep 08, 2005 at 17:01 UTC
As long as you're looking at perl code anyway, backticks should be somewhat familiar... $ prog.pl `ls file` [download] Of course, this is kinda silly - since you can just run "prog.pl file" to get the same output. But you may want to do something fancy: $ prog.pl `grep KEYWORD file* \| cut -f1 -d:` [download] Mind you, you can't embed backquotes in backquotes. Thankfully, you said "Linux" where I read "bash" over "sh" anyway. So we switch to something like this: `$ prog.pl $(grep $(getkeyword.pl foo) file* \| cut -f1 -d:)` [download] Remember: Unix isn't just a shell for running perl. :-)	[reply] [d/l] [select]
Re: Receiving Standard Input/Piped by halley (Prior) on Sep 08, 2005 at 16:51 UTC
(Update: mis-read your question, but still applies. Some text below updated to match your request.) If your task is simple or straightforward, you may just get away with the magic `<>` line iterator. This automatically pulls from each named file in turn, or from STDIN if there was none. This handles both of your cases magically. `# receive a list of filenames on stdin # (or from an intermediary file named on command line) # while (<>) { chomp; # process a filename in $_ }` [download] If you're new to Perl, I would also try to ween you away from "index loops" and toward iterating over the elements. You will find that you rarely NEED the integer representing the current position in a group of things. `# index loop for my $f (0 .. $#array) { my $g = $array[$f]; ... } # element loop for my $g (@array) { ... }` [download] -- `[ e d @ h a l l e y . c c ]`	[reply] [d/l] [select]