I don't think there's a good answer to this other than "Do it in C" but I thought I'd ask anyway. I'm reading in several hundred 20k line standard data text files, with space delimited numbers like so, though with up to 8 data columns: 
# time             data1     data2
0.000000          99.537     54.54
1.000000         100.273    121.54
2.000000          98.169    121.58
3.000000         105.835     99.66
4.000000          93.013      1.85

[download]
The time spent spliting the lines is about 10% of my program run time, so I was wondering if there was an easy way to speed it up. Just reading the files from disk seems to be about half my runtime, so no dramatic improvements possible. But look at the code, maybe a restructuring would be faster? Here's what I'm doing now: 
# Each data point is stored in a separate file, so they have
# to be joined first.  I figure the shell is faster at that
# than perl.  That and I wasn't excited about managing 
# n file handles at a time, though it wouldn't be too bad.
my @lines = `join file1 file2 file3 file4`;
foreach my $line (@lines) {
  my ($time, @data) = split /\s+/, $line;
  foreach my $datum (@data) {
    # unlock the secrets of the universe
  }
}

[download]

Lexicon

In reply to Speed of Split by Lexicon

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.