in reply to Re: how to remove duplicate based on the first column value
in thread how to remove duplicate based on the first column value

Which could also be done as the following one liner:
perl -e' while(<>) {print unless $s{(split /\|/)[0]}++;}' < infile > o +utfile
where infile is a file with the values to parse and outfile is where you want the results.

Indeed:

perl -i.bak -e' while(<>) {print unless $s{(split /\|/)[0]}++;}' infil +e
will edit infile in situ (putting backup in infile.bak).

Update: I have assumed that you are using a shell with redirection, such as bash. I have been told off for this sort of assumption before so best to make it clear.

--tidiness is the memory loss of environmental mnemonics

Replies are listed 'Best First'.
Re^3: how to remove duplicate based on the first column value
by revdiablo (Prior) on Jun 10, 2003 at 16:59 UTC

    Another nice perl command line switch is -n. This builds the while(<>) loop for you. Example:

    perl -i.bak -ne 'print unless $s{(split /\|/)[0]}++' infile

    Note: more of these can be found in perlrun.