I recently posted a question on how I could pipe many files into a single perl script, and received answers such as "use @ARGV", which worked great for small projects. The command line input worked just like I needed it to. The problem, however, is that it turns out I actually have far more files to deal with than I thought, on the order of 100,000 at a time.
Due to the size of the list in the command line, even when I use a wildcard, I get error output like this:
-bash: /usr/bin/perl: Argument list too long
Thus I have a problem. My input list is too long for my program to pipe in. What other solutions do I have? Is using @ARGV still a good idea, or do I have to look at another avenue?
UPDATE!!!:
I got my code to work! Thanks especially to Corion for recommending tye's sort function. Basically, this is how it played out:
@ARGV=glob my $pattern;
my @files=@ARGV;
my @sorted = @file[
map { unpack "N", substr($_,-4) }
sort
map {
my $key = $file[$_];
$key =~ s[(\d+)][ pack "N", $1 ]ge;
$key . pack "N", $_
} 0..$#file
];
@ARGV=@sorted;
while (<>) {
Do my function
}
if (eof(ARGV)) {
Do end of file cleanup
}
Using this format, I was able to still use the <> operator, while piping in a sorted ARGV, so my output came out like this:
file1
file2
....
file10
file11
This is exactly what I wanted. Thanks for all the help!
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.