in reply to Re^4: Search through multiple files and output results to new file
in thread Search through multiple files and output results to new file

So then the problem becomes, how do you distribute the file list across multiple grep instances?
There are several ways - the more you know about the file(s), the more options you have. However, that still uses grep.

Perhaps the OP can first elaborate on what's he's trying to achieve before offering solutions based on guesses.

  • Comment on Re^5: Search through multiple files and output results to new file

Replies are listed 'Best First'.
Re^6: Search through multiple files and output results to new file
by BrowserUk (Patriarch) on Aug 24, 2010 at 21:47 UTC
    Perhaps the OP can first elaborate on what's he's trying to achieve before offering solutions based on guesses.

    It seemed, and still seems to me, quite clear from the OP what he is hoping to do.

    Run the equivalent of  grep -h %search_string% *.txt > retuned_Results.sl in a manner than utilises his multiple cores to reduce the runtime of the query.

    Ie. Process the filelist generated from *.txt in parallel on all his cores, rather than serially on only one as the standard grep command would do; and collate the results into a single output file. I don't see any need for guesses.

    Whether this actually achieves a saving in runtime really depends upon the size of the files and complexity of the search parameters. It is actually quite difficult to compare like with like because of the affects of file system caching.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom