From a quick look, I don't see what is generating your output, but I do see a few things that will definitely slow you down.

1. You are reading filenames from the directory "/mnt/data/Programming", but opening those files in the current directory. If you are not in "/mnt/data/Programming", this should not open the files you intend.

2. You are modifying the directory you are reading by creating a file there. That is probably going to cause some confusion.

3. Since you are opening your output file with '>', you would overwrite it each time instead of accumulating output in that file. If you want to reopen each time, use '>>' instead.

4. You seem to be processing data one line at a time and also putting those lines in an array. I would probably only do one or the other.

I would probably get the list of files all at once and then process them from an array. For instance,

opendir ( my $dir, "/mnt/data/Programming")
   or die "Can't open directory: $!\n";
my @files = readdir( $dir );
closedir( $dir );
[download]

You can now process the files independently. The file open should really contain the path as well:

foreach my $file ( grep { -f "$path/$_" } @files )
{
    open( my $fh, '<', "$path/$file" )
       or die "Unable to open '$file': $!\n";

    while( <$fh> )
    {
        # process each line here.
    }
}
[download]

It's a good idea to check the names you get from readdir() to make sure they are files. You can't read directories as files (in general). Notice that I used lexical file and directory handles. Although not critical in this case, it will definitely save troubleshooting nasty problems in the future. Start on good habits early.

Update: Corrected file test to use path as shmem observed.

Yes, but...

The file open should really contain the path as well:

foreach my $file ( grep { -f $_ } @files )
[download]

...the file test should really contain the path as well ;-)

foreach my $file ( grep { -f "$path/$_" } @files )
[download]

if ($lines[$i] =~ /$regex[$i]/)

If $lines[0] doesn't match /file/ then you never will execute if body and never will increment $i, so you always would match $lines[0] against /file/.

Update: typo fixed

open (IN, "$file") || die $!
[download]

open (IN, "$path/$file") || die $!;
[download]

Update: but if you're executing the program from that directory anyway, then that's not your problem.

Just a minor note: You're opening & closing the output file handle repeatedly, which can be pretty slow if you must do it often. One way you could speed things up is to open the output file handle only once (if required) and close it only once at the end. Something like:

# First declare your file handle
my $OUT;
...
foreach ... {
   while ... {
      if ... {
         # Error found, so...

         # open the file if not open yet
         if (!defined $OUT) {
            open $OUT, '>', "$path/$file" or die "Uggabug! $!";
         }

         # and log the error
         print $OUT "Yer error message\n";
      }
   }
}
# finally, close it if you've opened it
close($OUT) if defined $OUT;
[download]

Now is there something obviously normal about this strange output...?

If your first debug print statement is uncommented you should see every line of every file you read but if only the second is uncommented you may well see something like the sample you provided.

You have two loops: an outer loop for each file in the directory and an inner loop for each line in the current file. On each iteration of the inner loop you push the current line onto @lines. So @lines eventually contains all the lines from all the files. Initially $lines[31] is undefined, so your second debug print statement will print nothing (but you should get a warning like "Use of uninitialized value in concatenation (.) or string at ..."), but after you have read 32 lines it will have a value and it will never change, so your second debug print will then print the same string each time through the loop.

Following is an example you might consider. It has a few advantages and some disadvantages:

• It slurps each file into a scalar which is reused on each iteration of the loop so it doesn't accumulate all the input lines.
• It uses a foreach loop to iterate over the list of search strings rather than an index into the array.
• It uses an RE to capture matched text: consecutive lines matching the given string and one preceding and one trailing line (like a context grep with one line of context). This RE is difficult to understand. This makes the loop tight/short but ultimately may be harder to understand and maintain. An alternative is a simple RE with additional code to concatenate matching and context lines.
• It organizes the results by matched string and by file.
• It uses a template to produce a complete HTML page - with HTML escaping of the file names and matched text.

my $DIR = "./";
opendir ( my $dir, $DIR) or die "Can't open directory $DIR: $!\n"; 
my @files = map { $DIR.$_ } readdir( $dir ); 
closedir( $dir );
foreach my $file ( grep { -f $_ } @files ) { 
    print "\$file is $file\n";
}
[download]