my @regexes = map qr{GET (.*$_.*) HTTP/1.1" 200 [0-9]}, @mask;
[download]

my (%count, @order);

while (<F>) {
  chomp;
  for my $rx (@regexes) {
    if (/$rx/) {
      my $bn = basename($1);
      $count{$bn}++ or push @order, $bn;
    }
  }
}
[download]

Update: in retrospect, there's probably no harm in producing a single regex, since looping over the regexes can't provide any additional hits. That is, if regex 1 finds a match and regex 2 finds the same match, then regex 1+2 together will provide the same results.

my $alts = join '|', map quotemeta, @masks;
my $rx = qr{GET (.*(?:$alts).*) HTTP/1.1" 200 [0-9]};
my (%count, @order);

while (<F>) {
  chomp;
  if (/$rx/) {
    my $bn = basename($1);
    $count{$bn}++ or push @order, $bn;
  }
}
[download]

holy mother of perl, this is truly jawdropping.

the script finishes in mere 12s... and you axed half of my script ;-)
as i don't really care about the order of the entries, another array axed. brilliant.

it is now clear for me that compiling regexes is more expensive than to fly to paris...

i would like to thank the others as well for their time and effort.

The two problems worked in tandem to screw you over bigtime. With more than one keyword being used, you had to compile a regex for EVERY line of the file and EVERY keyword. 200 lines and 2 keywords means 400 regex compilations (although you only really need TWO regexes). And using an array instead of a hash to keep track of uniqueness is, well, the road to madness.

---

Jeff japhy Pinyan, P.L., P.M., P.O.D, X.S.: Perl, regex, and perl hacker
How can we ever be the sold short or the cheated, we who for every service have long ago been overpaid? ~~ Meister Eckhart

foreach (@count) {
    if ($_->[0] eq $bn) {
        $_->[1]++;
        $found = 1;
        last;
    }
}

if ($found == 0) {
    push @count, [ $bn, 1 ];
}
[download]

++$count{$bn};
[download]

my $regex = join '|', @mask;
# Now you have a list of alternatives
$regex = qr<GET (.*\b(?:$regex)\b.*) HTTP/1.1" 200 [0-9].*>;
# now the regex is precompiled
[download]

if (/$regex/) {
    s/.*GET //;
    s/ HTTP.*//;
      :
      :
[download]

if (s/$regex/$1/) {
      :
      :
[download]

use strict;
use warnings;
use File::Basename;

my (@count, %seen);
my @mask = qw(/some/path/ some/other/path another/path);

open (FH, '<', "access.log") or die "Unable to open 'access.log' for r
+eading: $!";

while (<FH>) {
    chomp;
    for my $m (@mask) {
        my $regex = "GET.*" . $m . ".*HTTP/1.1\" 200 [0-9].*";
        if (/$regex/) {
            s/.*GET //;
            s/ HTTP.*//;
            my $bn = basename($_);
            push @count, $bn if ! $seen{$bn}++;
        }
    }
}
print "$_ = $seen{$_}\n" for @count;
[download]

Update: Following the suggestion of others to use qr// external to the loop which will increase the performance of this solution even more. If you can combine the regexes using Regexp::Assemble, there will be an additional boost.

HTTPD::Log::Filter is such a module, though I'm not sure it does much for the specific problems the OP was after. On the other hand, those seem to have been adequately addressed elsewhere, so I'll just throw in a plug for the module (which has worked well for me in the past) and leave it at that. :-)




---

If God had meant us to fly, he would *never* have given us the railroads.
    --Michael Flanders

If you have one mask, Perl knows the regex hasn't changed, so it won't recompile - hence you will compile only one regex, despite your 100,000 lines of code.

You can do two things - either switch the order of the loops , that is, for each mask, create a regex, then loop over the log file (first seeking back to the start of the file!), or create an array of compiled regular expressions, and loop over them.