I'd have thought that writing to a file and reading it back would have slowed me down, but it didn't!

There's only a difference in the filehandle types involved. In the first, the shell opens/closes $tmpfile, in the second, it opens/closes a pipe attached to the perl side pipe filehandle created by qx (which perl creates anyways), so it is no surprise there is no difference, specially if you are working with a SSD instead of an old washing machine type of disk drums (modern disks might hold the entire file in the controller cache, so perl can read the file even before it is allocated physically via magnetism).

It would be more interesting to benchmark the shell chain against a pure perl solution, in which case perl loses here. Why? Because allocating the necessary data structures in perl means some overhead, whereas the cut uniq sort utilities deal only with char arrays[1], are seasoned and thus optimized for their specific tasks.

Here's a file of ~132MB, one million records, created with 

$ perl -E 'say join",",map{int rand 1000000} 1..20 for 1..1000000' > s
+ample.csv

[download]

and a quick shot at timing: 

$ time cut -d"," -f 17 sample.csv | sort | uniq > out

real    0m4.391s
user    0m4.788s
sys    0m0.060s
$ time perl -F, -E '$s{$F[16]}++ }{ say for sort keys %s' sample.csv >
+ out

real    0m6.716s
user    0m6.668s
sys    0m0.048s

[download]

This could make a difference with huge files. I haven't looked at the memory footprint, which might be another clue for deciding for or against a (dogmatic) "pure perl solution".
The bias is always qw(laziness impatience hubris) in an order that fits best.

[1] afaik those utilities are UTF-8 agnostic

perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

In reply to Re: search/grep perl/*nix by shmem
in thread search/grep perl/*nix by Gtforce

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.