someone told me perl was extremely slow and it is wiser to use the shell.

Tell him he's an idiot and it's wiser to use the right tool for the right job (ok, leave that first part out if you want to be diplomatic).

...(They) wrote which ran in 3 secs in awk and c and 3 minutes in perl.

I used just a naive split in perl, and got about 25 seconds (perl) for processing 100,000 lines of 100 fields vs. about 10 seconds in awk. The perl took about 45 seconds if I used the "-a" and "-F" options instead of just using $tot += (split /,/)[99].

But what does that mean? For something insanely simple like this, I might use awk (and the 'C' just does not look insanely simple, so I don't think I'd do it that way). If I wanted to do something more complicated, like keeping group sums of combinations of other fields, and key it to some other nested data structures, and look up some other data in a database and some other data from the web, then it's wiser to get it done easily in your favorite scripting language.

FYI, the awk code (I have an old awk that only allows 99 fields):

#!/usr/bin/awk -f
                 
BEGIN { FS="," }
{ tot += $99 }    
END { print tot }

[download]

In reply to Re: fast way to split and sum an input file by runrig
in thread fast way to split and sum an input file by egunnar

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.