In general, I find that optimizing the database provides a benefit/cost¹ that's at least 10x greater than any Perl optimization. For example, if you have something like:

my $sth = $dbh->prepare_cached("SELECT foo,bar from table where id = ?
+");
for my $id (@ids) {
    $sth->execute( $id );
    my @results = $sth->fetchrow_array;
    $sth->finish;
}
[download]

my $placeholders = join ',', ('?') x @ids;
my $sth = $dbh->prepare_cached(
    "SELECT id,foo,bar from table where id IN ($placeholders)",
);

$sth->execute( @ids );
$sth->bind_columns( \my ($id, $foo, $bar ) );

while ( $sth->fetch ) {
    # Put values somewhere
}
$sth->finish;
[download]

That's both taking advantage of the best way to handle DBI as well as allowing the database to do more of the work. And, there's hundreds of similar restructurings that can provide up to 90% gains. The database will almost always be a major limiter of performance, especially if you're not doing it smart.

1. benefit/cost == The amount gained divided by (effort taken + effort to maintain). So, for example, if you were to replace all calls to a subroutine with the body of the subroutine, there would be a significant cost to do it (search+replace can do it, but testing's a bitch) plus a significant increase in maintenance (any change has to be applied with search+replace).

To satisfy my own curiosity, I benchmarked these. 800 elements in @id:

Benchmark: timing 50 iterations of execute_each, using_in...
execute_each: 227 wallclock secs ( 6.73 usr +  1.41 sys =  8.14 CPU) @
+  6.14/s (n=50)
  using_in: 17 wallclock secs ( 1.39 usr +  0.08 sys =  1.47 CPU) @ 34
+.06/s (n=50)
[download]

SheridanCat

BEGIN {
$short = '{ do_stuff ... }';
eval 'sub big { $something; '.$short.' $other_thing }';
}
[download]

Um, string-y eval under mod_perl can be a really, really bad idea, given that it can start a whole new interpreter instance. I'd suggest testing this thoroughly before attempting it in any production server.

Do not rebuke them with harsh words ... but rather lead them gently - with URLs - so that they may learn wisdom.

You already have an interpreter instance if you are running code under mod_perl. String eval under mod_perl does have the same downsides that it does in any other perl code though.

I know nothing about mod_perl. Nothing at all! But...

• You have subtracted the time spent in DBI->Connect() from the time spent in DB_connect(), to see what you would gain by using a macro, haven't you?

  Because unless calling DBI->Connect(...) is extremely fast, using a macro to substitute that call for DB_connect() is going to make very little difference.

• If DB_connect() is really profiling as a significant proportion of the runtime for one or more of your functions (run modes?), then you must be calling it a significant number of times--like every time you need a $dbh

  Why not call it once at the start of each function and assign it to a local variable?

  I realise that this increases your exposure to the lose of your DB connection (slightly?) but given your use of RaiseError, you are already handling the possibility of errors on individual calls to DBI, this would be just one more potential error.

  Is the probability of the connection dropping within any given function (as opposed to between them) really so high?

Beyond that, if you really need a macro facility, there are several possibilities:

1. The highly deprecated and discouraged -P option (see perlrun)

   This probably wouldn't work with mod_perl?

2. The alternative to -P mentioned in perlrun, Filter::cpp.
3. The universally disliked possibility of a custom source filter using Filter::Simple or Filter::Util::call packages.
4. Using a preprocessor as a part of your "build" process. That could be the C preprocessor or a dedicated tool like m4.

Note: I'm not recommending you use any of these--indeed, from what information you've posted it is hard to see how a macro facility would benefit you in any significant way--but it is good to have the information available even if only that you can make your own mind up with good conscience.

Memoize is used to change large/long-running subroutines into short/fast subroutines by caching results, and only for subroutines where there are no side effects, and all outputs can be computed solely from parameters. Reading from a database, for example, involves inputs that are not parameters, and thus fails this requirement.

The OP seems to be saying that there is an oft-called small routine that he would like to inline. That is something one would do with the "inline" keyword in C++, but in perl, you simply write a constant sub and hope that perl figures out that it's inlineable. Generally speaking, other than for constant, it hasn't sounded useful to me. So, these small routines are already small, and hopefully fast. I don't see that Memoize would be likely to be useful in a scenario where someone wants to remove even the overhead of a subroutine call.

Personally, I think what the OP is looking for is a source Filter that can take function definitions and put them into the code where they are called. For a single module, it shouldn't be too bad: just take everything inside the function (which should be very small) and put it in a do block, and take all the parameters that were being called, and put them at the top of the block as a localised @_ list.

Mind you, if I were that interested in speed, I'd be doing this whole thing in C or C++ anyway. Given that I'm already running perl, I'm not going to worry about the overhead of function calls. The speed benefit I, as the programmer, am getting is already huge. I'll just tell my boss he needs a faster 'puter. He'll probably save money that way anyway.

The speed benefit I, as the programmer, am getting is already huge. I'll just tell my boss he needs a faster 'puter. He'll probably save money that way anyway.

For the disbelievers, let's do a quick numbers game. The average cost to a company for an hour of a programmer's time (salary, benefits, lights, deskspace, etc) is around $70. For a top developer, that goes up to $100/hr or more. So, for a week's work (40 hours), that costs your company $2800. Now, a quick google found that you can get a dual-core Xeon server in a base configuration for about $3k from Dell. Let's say tricking it out puts that price up to about $7k. That's 100 hours, or 2.5 weeks of work.

In general, the server your app or db is running on is not a tricked-out dual-Xeon. I've seen performance boosts of 4-10x just by moving servers. Can you provide a similar boost from 2.5 weeks of coding, especially with no new bugs and no additional maintenance burden?

Additionally, you can often get a 2-4x boost just by realigning your database. Yeah ... code optimizations are often the last refuge of the incompetent. (Bonus points to whomever can identify the source of that misquote.)

---

My criteria for good software:

1. Does it work?
2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Don't know how well it'd play with mod_perl, but you might look at POE::Preprocessor to at least get some ideas.

mirod uses this trick Speeding-up OO accessor methods in XML::Twig.