In general, Class::DBI (or any other O/R mapper) is going to have poor performance when used for bulk loading. To get really good performance on that stuff, you need to use your database's bulk loading tools.

The app walks through a database, pulling "active" rows (rows needing work). It pulls some related info, does a computation, then passes the CDBI objects off to TT for rendering as an output file. As the file is large, rather than pass a big array of Foo to TT, each is passed one by one, and TT appends the result to an open file handle.

Apologies for the obfu names in the code. Here's the loop that does all the work and takes all the time:

# obfu'd code, with some addntl comments


# NB: there are many many of these
my $it = Zmain::Foo->search( client => $client, status => 'active' ); 

    while ( my $Foo = $it->next ) {

# NB this is a master table of Things.  A Foo is a specific kind of Th
+ing.  The master Thing table keeps ids and common info straight.

        my $thing = Zmain::Thing->retrieve($Foo->Foo); 

# NB not sure if this assert wastes time stringifying thing for the fa
+ilure message if the assertion passes?        

        assert( $thing, "found thing=" . $thing->thing );

    my $info        = $Foo->Bar->info; 

        my $computed = $tm->compute( $info, $thing );

        $tt->process(
            'template.tt',
            { Foo => $Foo, computed => $computed },

# just handle one at a time

            sub { print $fh shift } 
        );

}
[download]

Well, you are right about the assert() call being wasteful. It's going to hit all of the subs that are in the top of your dprofpp output. However, if you really want to speed this up, you need to modify your approach your drastically than just removing that.

What you have here is really a join. Instead of doing multiple fetches, you could do one large fetch, and then construct your objects from it. Basically, I would do one query that pulls back all the columns you need from both tables (you could make this a custom SQL method of a Class::DBI object if you want to), and then feed the results to the construct() method on your classes. See the "QUICK RETRIEVAL" section of the Class::DBI docs for more on this.