in reply to Improving the efficiency of code when processed against large amount of data

To add to what the other monks have said ...

There's an error and at least one weirdness in your code:

  1. the next line works by accident. It's checking if $filename in the current directory is a directory or not. If there happens to be a directory in the current directory that has the same name as a file in your summary spr directory, it will be skipped.
  2. your map-in-void-context looks like it might just be trying to act as chomp. If so, it's just chomp(@{$sspr_hash{$file_name}} = <FH>); with no map afterwards.
  3. grep works on lists but you're only using it on a single item. Valid but strange. Typically people would just do $line =~ /$old_schema/i
Also, I'm not sure why you've divided your task into two parts like this. Is there a reason you're reading the contents of the files before you do any processing? If not, I'd just do it all in one go like this:
#!/usr/bin/perl use warnings; use strict; my $ssprdir = "/apps/inst1/metrica/TechnologyPacks/ON-SITE/summaryspr/ +"; opendir(my $sspr, $ssprdir) or die "Can't read $ssprdir - $!\n"; my @files = grep { ! -d $_ } map { "$ssprdir/$_" } readdir $sspr; closedir $sspr; for my $file (@files) { unless (open(my $fh, "<", $file)) { warn "Unable to open $file - $!\n"; next; } # ... while (<$fh>) { next unless /$old_schema/i; # ... print "$old_schema|$new_schema_str|$found_status|$migratestr|$ +rename_str|$file\n"; } close $fh; } __END__

Also, if $old_schema is really supposed to be a string and not a regular expression, you might consider using index instead of engaging the RE engine.