Monks I come to you seeking your wisdom once again. I am having a bit of a memory management issue I was hoping you could help me with. I have written a script that produces a daily report on a system scan it does. It first reads in a benchmark file from the scan the day before into a hash. It then runs the daily scan dumping the info it finds into the new benchmark file and also compares the information to the data from the day before in the hash.

The problem I am running into is the data being read into the hash can be several hunder megabytes in size. I need to find a more efficent way to handle this data. Here is an example of the code I have written.

use strict;

sub _read_benchmark {
       my %yesterday;
       open BENCH, "benchmark_file" or die "$!";
       while (<BENCH>) {
              chomp;
              $yesterday{$_}++;
       }
       close BENCH or warn "$!";
       return \%yesterday;
}

sub _scan_system {
       my $yesterday = shift;
       my %today;
       open BENCH, "> benchmark_file" or die "$!";
       open IN, "find / $search_files -print |" or die "$!";
       while (<IN>) {
              chomp;
              print BENCH "$_\n";
              if (exists $yesterday->{$_}) {
                     delete $yesterday->{$_};
              } else {
                     $today{$_}++;
              }
       }
       close IN or warn "$!";
       close BENCH or warn "$!";
       return \%today, $yesterday;
}

sub _print_report {
       ...
}

my $yesterday = _read_benchmark;
my ($today, $yesterday) = _scan_system($yesterday);
_print_report($today, $yesterday);
[download]

I believe there is a way I can tie the benchmark file to a hash and that would probably be a huge improvement, but I am not sure how to do that. Any suggestions on how to make this less of a memory hog would be very appreciated. Thanks!

-Prime

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.