glwtta has asked for the wisdom of the Perl Monks concerning the following question:

IO::All is one of the most convenient modules I've found in a long time. So I was a little dismayed to come across this little problem.

I have a file which is roughly half a GB (11 million lines), simply doing:

my $line; while ($line = <>){ print "$.\n" unless ($. % 10000); # to keep track of the progress }
takes about 7.5 seconds.
With
my $in = FileHandle->new($ARGV[0]); my $line; while ($line = $in->getline){ print "$.\n" unless ($. % 10000); }
takes about 23 seconds (same for IO::File); whereas this:
use IO::All; my $in = io($ARGV[0]); my $line; while ($line = $in->getline){ print "$.\n" unless ($. % 10000); }
takes over 10 minutes!

Are others seeing this sort of performance? Is this to be expected?

If it makes any difference, I am running perl 5.8.4 on RedHat Linux 7.3 (reiserfs)

Replies are listed 'Best First'.
Re: IO::All speed issue
by eserte (Deacon) on May 11, 2004 at 15:27 UTC
    You either get convenience, or speed, but never both (well, almost never...).
Re: IO::All speed issue
by duff (Parson) on May 11, 2004 at 15:34 UTC
    Magic and method dispatch conspire to make it slow. Use less magic and it will be fine. Good, fast, cheap--pick any two, but not that middle one if you're using IO::All :-)