Do you need to retain the values outside of the loop? When processing 8 million lines, I wonder if you put all of that data into a hash, as that is likely to run out of memory.

If the parsed data is "healthy" - as in all lines have the same number of fields, neatly separated by single TAB characters - and you only need the fields *during the iteration*, you could consider using Text::CSV_XS in combination with bind_columns. That will re-use the assigned variables for each line and /might/ save a lot of time.

--8<---
use strict;
use warnings;
use Benchmark qw(cmpthese);

my @data = ("aaaaaaaaaa") x 100;
my $line = join "\t" => @data;
my $data = join "\n" => map { $line } 0 .. 50000;

use Text::CSV_XS;
my $csv = Text::CSV_XS->new ({ sep_char => "\t", binary => 1, auto_dia
+g => 1 });

sub bsplit
{
    open my $fh, "<", \$data;
    while (<$fh>) {
        chomp;
        my @aa = split /\t/ => $_;
        }
    } # bsplit

sub csvxs
{
    open my $fh, "<", \$data;
    my @aa = ("") x scalar @data;
    $csv->bind_columns (\(@aa));
    while ($csv->getline ($fh)) {
        }
    } # csvxs

cmpthese (10, {
    "split"    => \&bsplit,
    "csv_xs"    => \&csvxs,
    });
-->8---

=>

         Rate  split csv_xs
split  1.60/s     --   -27%
csv_xs 2.21/s    38%     --
[download]

Note that the difference is *completely* depending on what your data looks like!

my @data = ("aaaaa") x 10;
my $line = join "\t" => @data;
my $data = join "\n" => map { $line } 0 .. 500000;

=>
         Rate csv_xs  split
csv_xs 1.17/s     --   -13%
split  1.34/s    14%     --
-------------------------------------------------
my @data = ("aaaaa") x 500;
my $line = join "\t" => @data;
my $data = join "\n" => map { $line } 0 .. 50000;

=>
       s/iter  split csv_xs
split    2.93     --   -57%
csv_xs   1.25   134%     --
-------------------------------------------------
my @data = ("aaaaaaaaaaaaaaaaaaaaaaaaaaa") x 500;
my $line = join "\t" => @data;
my $data = join "\n" => map { $line } 0 .. 50000;

=>
       s/iter csv_xs  split
csv_xs   3.74     --    -5%
split    3.55     5%     --
[download]

Even running the same on a different machine might change the results. Benchmark if things get important, but bench where it matters: do not draw conclusions on a benchmark run on a 64bit non-threaded non-debugging perl on a 64core machine with 128Gb of RAM to have the same result on an AIX 5.2 with 32bit threaded debugging perl with only 1 CPU and 2 Gb of RAM available to the process.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.