It would seem best to put all your 'data_gathering' processes in one module. There has to be a lot of commonality in the code for each company.

In the module you might have the following subs.

Now you need to tell the module which process combination to use. For this you could use a hash, outside the module, that defines the process for each company. That way you don't need to mess with the module when you add a new company provided, of course, that the required sub exists in the module.

use Crunch;

%process = (
    'WalMart' => 'A:A:A',
    'ExxonMobil' => 'B:D:A',
    ...
    'CiscoSystemsInc' => 'C:D:A',
);

sub do_em {
  my $path_to_companies = shift;
  opendir $dh, $path_to_companies or die "Can't opendir $path_to_compa
+nies: $!\n";
  while ( my $company = readdir( $dh ) ) {
    next if $company ~= /^\./;
    my $dir = catdir( $path_to_companies, $company );
    next unless -d $dir;
    Crunch::do_it(
        dir => $dir
        company => $company
        process => $process{$company);
    );
  }
}

[download]

In reply to Re: Thorny design problem by nedals
in thread Thorny design problem by tlm

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.