OT: Find unique values in a column

rahulme81 has asked for the wisdom of the Perl Monks concerning the following question:

I made a grep

grep -w BG file.txt

on my configuration file which produces me an output as below

 
DATABASE_NAME                  DB_ABB           DB_ID             DB_D
+IR
============================================================
DATA_ARCHIVE                   BDAT    -    -   BG      -         sas 
DATA_CORP                      BDAC    -    Y   BG      bg        sas 
DAATA_ARCHIVE                  BDAA    -    -   BG      -         hado
+op
DATA_CONS                      BDAC    -    -   BG      -         hado
+op
DATA_APPS                      BDAP    -    -   BG      ba        misc
+  
DATA_HISTORY                   BDAH    -    -   BG      -         hado
+op
[download]

What I need to achieve now is, based on DB_ID column value "BG" (which i grep earlier) I need to print the DB_DIR unique values

output should be unique DB_DIR values as below

sas

hadoop

misc

Please advice

Comment on OT: Find unique values in a column Select or Download Code

Replies are listed 'Best First'.
Re: OT: Find unique values in a column by choroba (Cardinal) on Nov 30, 2015 at 15:26 UTC
This is a FAQ. `echo 'DATA_ARCHIVE BDAT - - BG - + sas DATA_CORP BDAC - Y BG bg sas DAATA_ARCHIVE BDAA - - BG - hado +op DATA_CONS BDAC - - BG - hado +op DATA_APPS BDAP - - BG ba misc DATA_HISTORY BDAH - - BG - hado +op' \ \| perl -lane 'print $F[6] if ! $seen{ $F[6] }++'` [download] `-l` adds a newline to print (see perlrun) `-n` processes the input line by line `-a` splits the input line on whitespace into the `@F` array The seventh columns is printed if it hasn't been seen yet. ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l] [select]
Re^2: OT: Find unique values in a column by rahulme81 (Sexton) on Nov 30, 2015 at 15:32 UTC
thank you !!! Is there any other way I can achieve with awk or unix one liner commands?	[reply]
Re^3: OT: Find unique values in a column by choroba (Cardinal) on Nov 30, 2015 at 15:42 UTC
Of course, but you don't ask for awk or unix in a Perl forum, do you? `... \| rev \| cut -f1 -d' ' \| rev \| sort -u` [download] ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l] [select]
Re^3: OT: Find unique values in a column by Laurent_R (Canon) on Nov 30, 2015 at 20:42 UTC
Is there any other way I can achieve with awk or unix one liner commands? Well, `perl -lane 'print $F[6] if ! $seen{ $F[6] }++' file.txt` [download] is sort of a Unix one-liner, isn't it? `;-)`	[reply] [d/l] [select]