Working backwards from the end product, which is a unified, well organized database of test results, you need one module that can produce output data in that form: it knows what the common fields are that need to be filled in for each entry that it creates, so it simply provides a function that can be used by its caller, with a consistent list or hash structure of information needed for an entry.

Then, for each distinct format of input file from your various sources of "raw" test data, you will need a module (or just a particular function/subroutine) that is designed to read the given input format, extract the pieces of data that are needed to produce the common output structure, and return that structure so it can be passed to the output module described above.

Finally, the last piece you need is a "controller" script, which can figure out what kind of file it's going to deal with, pass that file to the appropriate parsing subroutine, get back the common data structure, and pass the structure to the output module.

So, start by writing the output module; then pick one of the many input formats, and write a subroutine that will read one of those files, convert the file contents to the data structure needed for output, and send back that structure as its return value. Then write the controller, whose job it is to identify file names of a given type, and iterate over them: pass the file to the parsing routine, get back the common data structure, pass that structure to the output module, move on to the next file.

Once that's working, it's just a matter of adding more subroutines as needed for each distinct type of input file, and making sure the controller script knows how to sort things out properly, so that the right parser is called for each type of input test data. I would see it as three script files:

• CommonOutput.pm : module to produce the intended output form of the data
• TestParsers.pm : module of subroutines for the various input formats
• test_reporter.pl : executable script that uses those modules

The executable script could look sort of like this:

!#/usr/bin/perl

use strict;
use CommonOutput;
use TestParsers;

( @ARGV == 1 and -d $ARGV[0] )
    or die "Usage: $0 pathname\n  pathname : directory containing file
+s to process\n";

chdir $ARGV[0] or die "chdir $ARGV[0] failed: $!\n";

# let's suppose that TestParsers can tell which parsing
# function is needed based on the path (because the path
# identifies both the source and the type of content):

my $parser = TestParsers->new( $ARGV[0] );
my $emitter = CommonOutput->new();

for my $infile ( <*.txt> ) {  # or whatever works for your files
    if ( open( IN, "<", $infile )) {
        local $/ = undef;
        $_ = <IN>;
        close IN;

        emitter->print( $parser->( $_ ));
    }
    else {
        warn "unable to open $infile: $!\n";
    }
}
[download]

The CommonOutput module should be pretty simple. As for the TestParsers module, its "new" method takes a path, decides which parsing function is needed, and returns a reference to that particular subroutine. The sub refs can be put into a hash, keyed by path:

package TestParsers;

sub file_type_A
{
    local $_ = shift;  # parameter is full text of file
    chomp;

    my @field_labels = qw/date result tester/; # common pieces of data
    my %field_content;
    @field_content{@field_labels} = ( split /\|/ )[1,4,6];

    return \%field_content;
}

# more subs for other formats... (B, C, etc)

sub new {
    my $sub_hash = ( 'Gearbox/Integration' => \&file_type_B,
                     'Gearbox/Assembly' => \&file_type_A,
                     ...
                     'Brakedrum/Performance' => \&file_type_A,
                     ...  # diff. paths might use same func
                    );

    return $subhash{$_[0]}
}
[download]