in reply to Not sure what I am doing wrong

The first thing you should do is to get the close() calls in the right context...
#!/usr/bin/perl use strict; use warnings; use File::Find; my $DIR = "/tmp/logs/"; find(\&queries, $DIR); sub queries() { if ( -f and /^ftr001/ ) { print "----------------------------------------------\n"; print "Searching File:", $_, "\n"; print "----------------------------------------------\n"; open(UNZIP, "gunzip -dc < $_ |") or die "gunzip: $!"; open(RECORDS, "/tmp/logs/records.txt") or die "Could not open +records file: $!"; my @REC=<RECORDS>; chomp @REC; close RECORDS; while ( <UNZIP> ) { my $EACH_LINE = $_; foreach my $REC_QUERY (@REC) { print "Searching for current record: $REC_QUERY\n"; if ( $EACH_LINE =~ /\($REC_QUERY\)/ ) { print "Querying for record: $REC_QUERY\n"; print "$EACH_LINE\n"; } } } close UNZIP; } }
...in that way you'll be closing the files at the right time i.e. after you've finished with them. Otherwise you'll encounter problems...

After that, if I've understood you correctly, the simplest approach would be to convert the sought after records into an RE and use that against each file. Moreover, there's a marked performance hit by opening the records file once for every directory - c/w once at the outset - something along the lines of the following (untested) snippet...

#!/usr/bin/perl use strict; use warnings; use File::Find; my $DIR = "/tmp/logs/"; find(\&queries, $DIR); open(RECORDS, "/tmp/logs/records.txt") or die "Could not open records +file: $!"; local $/ = undef; # Ignoring binmode as its a text file my $RE = join '|', <RECORDS>; close RECORDS; sub queries() { return unless -f and /^ftr001/; print "----------------------------------------------\n"; print "Searching File:", $_, "\n"; print "----------------------------------------------\n"; open(UNZIP, "gunzip -dc < $_ |") or die "gunzip: $!"; while ( <UNZIP> ) { next unless /$RE/; print $_; } close UNZIP; }

A user level that continues to overstate my experience :-))

Replies are listed 'Best First'.
Re^2: Not sure what I am doing wrong
by learningperl01 (Beadle) on Nov 24, 2008 at 16:11 UTC
    Thanks for the quick reply. I used the code above which looks to be correct and I get the following error message? Not 100% sure why that error is coming up.
    Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11802. Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11803. Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11804. Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11805. Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11806. Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11807. Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11808. Use of uninitialized value in regexp compilation at dns.pl line 23, <U +NZIP> line 11809.
    Did you mean to put the following out of the sub()?
    open(RECORDS, "/tmp/logs/records.txt") or die "Could not open records +file: $!"; local $/ = undef; # Ignoring binmode as its a text file my $RE = join '|', <RECORDS>; close RECORDS;
    I tried moving the stuff above inside the sub and printed the $RE var and got the correct records but got no results. Not sure what the problem is. All in all what I am trying to figure out if there is a faster way to search for each of the 100 key entries in my file against all the logs files. The current problem(if thats what you want to call it) is that it takes almost 1 hour to search for 100 keywords against each of the log files which are about 15-18 MB in size. Thanks again for all the HELP!!
      Yep, moving that block saves an awful lot of time - as I said, with the block moved, you open the records.txt once only - c/w once per file.

      As far as the uninit value is concerned, try...

      use File::Find; my $DIR = "/tmp/logs/"; open(RECORDS, "/tmp/logs/records.txt") or die "Could not open records +file: $!"; local $/ = undef; # Ignoring binmode as its a text file my $RE = join '|', <RECORDS>; close RECORDS; find(\&queries, $DIR);
      Oops, sorry, my fault - as shown, the $RE lexical variable wasn't defined until after use (by the queries sub).

      A user level that continues to overstate my experience :-))