Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

[off-site] Bash + Perl oneliners basics

by chanio (Priest)
on Mar 17, 2005 at 05:20 UTC ( #440268=snippet: print w/replies, xml ) Need Help??
Description: Learning the power of working from the LINUX command line, combining multi-purpose Bash commands with Perl. Explained by one of the best RedHat perl programmers.(note: I apreciate all the following wise comments, but read the first paragraph of the first article, where he explains that he is going to do some unreadable code, inscrutable code, and disposable code :)
quote:"...if perl is the magic we brew here, then bash is the cauldron we brew it in")

Two articles:

How I learned to stop worrying and love the command line by Chip Turner, part 1 (
How I learned to stop worrying and love the command line by Chip Turner, part 2 (

I hope you'll enjoy it as I did.

There are also other articles that might be used as incentive for further reading about oneliners. One is a nice story writen with the Humphrey Bogart films' fashion but about a perl case. Read this Ben Okopnik (a oneliner expert) story: Perl One-Liner of the Month: The Adventure of the Spicy Blonde( There are also newer 'adventures' at the original LG's site ( (NOTE: see also this story... to round all up)

cat /var/log/httpd/access_log | perl -l -a -n -e 'print $F[6]' | sort 
+| uniq -c | sort -n | tail -10
Replies are listed 'Best First'.
Re: [off-site] Bash + Perl oneliners basics
by merlyn (Sage) on Mar 17, 2005 at 07:06 UTC
    cat /var/log/httpd/access_log | perl -l -a -n -e 'print $F[6]' | sort +| uniq -c | sort -n | tail -10
    Hmm. A Useless Use of Cat, using Perl like it was awk, and then chaining together a few other tools like forking is free. Hmm.

    I'd probably have written that as:

    @ARGV = qw(/var/log/httpd/access_log); my %count; while (<>) { my ($f) = (split)[6]; $count{$f}++; } my $n = 0; for (sort {$count{$b} <=> $count {$a}) { print "$_\n"; last if ++$n >= 10; }
    I bet mine runs with 1/4th the CPU.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

      I bet mine runs with 1/4th the CPU.
      Except that for small to medium sized files, it doesn't matter and the additional programming (and debugging) time dwarves the running time. And for really long files, your program might actually be slower, or even fail to finish as it will consume significant amounts of memory. The elegant one-liner, consisting of several tools that do one thing well won't suffer from memory problems, as 'sort' knows when to switch to using temporary files.

      Having said that, I would have written the one-liner as:

      awk '{print $6}' /var/log/httpd/access_log | sort | uniq -c | sort -n | head -10

      I know you're just tossing that code off quickly, but I'm curious to know why you chose to write:

      while (<>) { my ($f) = (split)[6]; $count{$f}++; }

      ...rather than...

      while (<>) { $count{(split)[6]}++; }

      It makes me wonder if there's some robustness principle at work that eludes me. And of course, there is even...

      $count{(split)[6]}++ while <>;

      ... but then we are getting into the realms of the cryptic, and I don't seen a more concise way of printing the top N values that doesn't sacrifice economy.

      - another intruder with the mooring in the heart of the Perl

      TBH I'd lose the Perl altogether:

      awk '{ file[$7]++ } END { for ( v in file ) print file[v], v }' /var/ +log/httpd/access_log | sort -n | tail +10
      I'm sure you could lose the rest of the pipe too but I never got my head around AWK's asort() for cases like this.


      chaining together a few other tools like forking is free
      By your reasoning, doing anything with the computer is not free, so why try at all? In my opinion, the cost of something like this is akin to the cost of gum balls: individually, they're so cheap that almost no one has a hard time justifying quantities of less than 100. And if you find yourself arguing with someone over the cost of a gum ball or 1000, just walk away. Your time is better spent.


      Feel the white light, the light within
      Be your own disciple, fan the sparks of will
      For all of us waiting, your kingdom will come

Re: [off-site] Bash + Perl oneliners basics
by chanio (Priest) on Mar 18, 2005 at 06:32 UTC
    Thanks, I got that!

    But what draws my attention in these articles is that I never see perl working together with other big ones like awk and bash (, etc.) in LINUX how-tos.

    These articles show something from the real LINUX distributions building job experience. There, perl has a significant place and combines well with all the other 'biggies' (just like Mc Giver's swiss-knife). This guy is writing about that point of view. But normally, people just mention perl as a tool or all the others without perl. And it is nice and fair to see them working as a group, don't you think so?

    .{\('v')/}   C H E E R   U P !
     _`(___)' ___a_l_b_e_r_t_o_________
    Wherever I lay my KNOPPIX disk, a new FREE LINUX nation could be established.
Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: snippet [id://440268]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (3)
As of 2022-08-19 04:37 GMT
Find Nodes?
    Voting Booth?

    No recent polls found