in reply to Re^3: Search through multiple files and output results to new file
in thread Search through multiple files and output results to new file

Oh. I never got the impression that the OP wanted this. I fail to see why that's a benefit.

If you start two concurrent instances of grep with a wildcard filespec, (win or *nix), then both instances will expand that wildcard to the same list.

So then the problem becomes, how do you distribute the file list across multiple grep instances?

Windows doesn't allow two processes to open the same file?

It does, but not by default. And the permissions required are not available from perl.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy
  • Comment on Re^4: Search through multiple files and output results to new file

Replies are listed 'Best First'.
Re^5: Search through multiple files and output results to new file
by JavaFan (Canon) on Aug 24, 2010 at 21:16 UTC
    So then the problem becomes, how do you distribute the file list across multiple grep instances?
    There are several ways - the more you know about the file(s), the more options you have. However, that still uses grep.

    Perhaps the OP can first elaborate on what's he's trying to achieve before offering solutions based on guesses.

[reply]
      Perhaps the OP can first elaborate on what's he's trying to achieve before offering solutions based on guesses.

      It seemed, and still seems to me, quite clear from the OP what he is hoping to do.

      Run the equivalent of  grep -h %search_string% *.txt > retuned_Results.sl in a manner than utilises his multiple cores to reduce the runtime of the query.

      Ie. Process the filelist generated from *.txt in parallel on all his cores, rather than serially on only one as the standard grep command would do; and collate the results into a single output file. I don't see any need for guesses.

      Whether this actually achieves a saving in runtime really depends upon the size of the files and complexity of the search parameters. It is actually quite difficult to compare like with like because of the affects of file system caching.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom