Do you have any pointers to practical code generation (other than compiler optimizers)?

"Practical" is a value judgement, an area where my judgement seems to vary widely with the concensus, so you'll have to make up your own mind.

In terms of the OP's question, I consider this practical. It runs a gnat's appendage slower than a hardcoded unrolled loop, but is 'generic':

sub aveAoA {
    my( $ref, $n ) = @_;
    my @sums;
    my $code = qq[ for my \$i ( 0 .. \$#\$ref ) { ];
    $code .= qq[ \$sums[ $_ ] += \$ref->[ \$i ][ $_ ];] for 0 .. $n -1
+;
    $code .= '}';

    eval $code;

    $sums[ $_ ] /= @{ $ref } for 0 .. $n -1;
    return @sums;
}

our $M ||= 1e6;
our $N ||= 4;

my @data; 
$#data = $M -1;
$data[ $_ ] = [ map int(-10+rand 20), 1.. $N ] for 0 .. $#data;

my @sums = aveAoA( \@data, $N );
[download]

The eval'd code is determanistic, and in my opinion totally 'safe'. Effectively, it is no different from selectively loading an appropriate subroutine (via import) at runtime except that it avoids having to have pre-written subroutines for every possible contingency.

For existing, well-known and popular modules that already use runtime code generation, see CGI, Catalyst, Template::Toolkit, Moose (though you have to drill down a few levels), Test::More etc.