perl printf vs C fprintf performance

decnartne has asked for the wisdom of the Perl Monks concerning the following question:

I'm probably missing something obvious... Why do many C fprintf statements take around 1 second (wallclock) and the equivalent perl printf statements take 4 seconds? Consider (ignoring for now some of my perhaps idiomatic failings):

#!/usr/bin/perl -w

use strict;

my $outf = "/tmp/outf.p.out";

open OUTF, ">$outf" or die "ERROR: could not open file $outf - $!\n";

foreach my $i ( 1..1000 )
{
    foreach my $j ( 0..255 )
    {
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
    }
}

close OUTF;

exit 0;

__DATA__

/*  C (somewhat?) equivalent of the perl script above  */
#include <stdio.h>

int main ()
{
    char *outf = "/tmp/outf.c.out";
    FILE *fd;
    int i,j;

    if ( !( fd = fopen( outf, "w" ) ) )
    {
        printf( "ERROR: could not open file %s\n", outf );
        exit(1);
    }

    for ( i=1;i<=1000;i++ )
    {
        for ( j=0;j<=255;j++ )
        {
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
        }
    }

    fclose( fd );

    exit(0);
}
[download]

My configuration is as follows:

hostname% uname -a
SunOS hostname 5.9 Generic_117171-07 sun4u sparc SUNW,Sun-Fire-V440
hostname% perl -v

This is perl, v5.6.1 built for sun4-solaris-64int

Timed results are:

hostname% time ../bin/ftest
1.0u 0.0s 0:01 85% 0+0k 0+0io 0pf+0w
hostname% time ./ftest.pl
4.0u 0.0s 0:04 86% 0+0k 0+0io 0pf+0w

(... for the curious, I have ported a C program which generates our DHCP table to perl, thus the pseudo formatting of the string...)

Taking out the hex formatting results in a faster run time (duh!), but still slower than equivalent C fprintf().

Thanks in advance for any info...

decnartne ~ entranced

Comment on perl printf vs C fprintf performance Download Code

Replies are listed 'Best First'.
Re: perl printf vs C fprintf performance by moot (Chaplain) on Mar 14, 2005 at 19:59 UTC
The unix 'time' command is a very unreliable indicator of program performance. Included in those 4 wallclock seconds, for instance, are the time taken to start the perl interpreter; the time taken to read, parse, and compile your script; and the time taken to unload everything at the end, to name but a few. You might want to look into Benchmark to see how your perl script is really performing. There are also several other great references available with a few moments googling, with more general benchmark advice.	[reply]
Re^2: perl printf vs C fprintf performance by decnartne (Beadle) on Mar 14, 2005 at 20:05 UTC
agreed - Benchmark probably would have been more accurate for times... I did actually use it in the original script to debug where my performance hit was. I found that the actual writing (the 8 or so printf statements) took ~8 seconds whereas the C app took ~2 All that aside - I'm still somewhat at a loss (surprise!) as to why writing ~ 1M lines in a file w/ C fprintf is so much faster... decnartne ~ entranced	[reply]
Re: perl printf vs C fprintf performance by brian_d_foy (Abbot) on Mar 14, 2005 at 20:07 UTC
You're not simply timing the print statements, but also the perl overhead (which gets less and less significant the larger your program gets) as well as the compilation time (which can continue to be significant). To be fair to perl, you'd have to also add in the time to compile your C program and whatever else you have to do to make it an executable. Still, you should never be surprised that C is faster than perl. :) -- brian d foy <bdfoy@cpan.org>	[reply]
Re^2: perl printf vs C fprintf performance by decnartne (Beadle) on Mar 14, 2005 at 20:11 UTC
...understood. I guess, I'm just surprised that C was that much faster in this instance. The replacement script fairs pretty well processing all of the records from the DB, etc. However, I am just a little curious how printf vs fprintf differ enough for a ~6 second difference... decnartne ~ entranced	[reply]
Re: perl printf vs C fprintf performance by hv (Prior) on Mar 15, 2005 at 01:22 UTC
I used the following code to benchmark, in an attempt to reduce the comparison solely to the actual printf calls: `use warnings; use Benchmark qw(cmpthese); open OUTF, '>', '/dev/null' or die "/dev/null: $!\n"; cmpthese(0, { perlprint => q{ for my $j (0 .. 255) { printf OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $j, +$j; } }, Cprint => q{ for my $j (0 .. 255) { C_print(OUTF, $j); } }, }); use Inline 'C' => <<INLINE; void C_print(FILE fp, int j) { fprintf(fp, "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", j, j); } INLINE` [download] This consistently reported the C code as being about twice as fast as the perl code (on my perl-5.8.5 installation), which seems about reasonable to me given all the extra layers of indirection the perl code has to wade through. One of the differences between perl-5.6.x and perl-5.8.x is that the 'perlio' abstraction is now the default, and with perlio enabled we get to take advantage of some of the lower-level I/O mechanisms available to the C library fprintf. It is possible that switching to a perl built with 'perlio' will get you a factor closer to this - check for "perlio=define" in the `perl -V` output. Hugo	[reply] [d/l] [select]
Re^2: perl printf vs C fprintf performance by decnartne (Beadle) on Mar 15, 2005 at 14:23 UTC
...thx for the explanation decnartne ~ entranced	[reply]
Re^2: perl printf vs C fprintf performance by Leotrillo (Initiate) on Nov 02, 2009 at 21:58 UTC
Hi, my name is Leo an i have the same problem, is very slow to open some document to write in it. you comment "It is possible that switching to a perl built with 'perlio' will get you a factor closer to this - check for "perlio=define" in the perl -V output." Can you put some examples to see how i can to accelerate this r/w docs in perl? thank you.	[reply]
Re^3: perl printf vs C fprintf performance by Anonymous Monk on Nov 03, 2009 at 04:32 UTC
useperlio is an option you set when compiling/installing perl http://search.cpan.org/dist/perl/INSTALL `$ perl -V \|grep -i perlio useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=und +ef USE_LARGE_FILES USE_PERLIO USE_SITECUSTOMIZE $ perl -V:useperlio useperlio='define';` [download]	[reply] [d/l]
Re: perl printf vs C fprintf performance by sh1tn (Priest) on Mar 14, 2005 at 21:54 UTC
One strict criteria: `%strace -c ./ftime %strace -c ./ftime.pl` [download] Thus one can see which spends what time and where (as far as system calls are concerned).	[reply] [d/l]