One of the problems with using backticks to run system executables is that the errors are not returned to your perl program. You would have to combine the stderr of grep, or awk, or whatever is throwing the error, with its stdout, and then parse what you get back.

1,000 files times 400 lines times four files that each line could appear in is really not that much data. It sure seems like you could read it all into memory and process your search and match, neatly and easily using Perl's built-in tools, and it wouldn't be unacceptably slow.

I would definitely try that and benchmark it first, before using my perl script to run a bunch of shell commands that are piping to each other.