Hi, We have a .NET application to be tested in Perl. We are working on .NET 4.0 version and on the Windows 7 OS. My requirement is to identify the duplicate records in a text file. We have nearly 1000 of these files to be done and time frame is very very less. A sample line of data in the file : ABC Mutual Funds |00000111|001110407033|011000073|FRF|CG12088886S11086: ABC Mutual Funds|665.26 
 :
#!/usr/bin/perl

use strict;
use warnings;
use File::IO;

my %no_dupes;
 
while (<>) {
 chomp;
 $no_dupes{$_} = $. if (! exists $no_dupes{$_});
 }
 
$result_file="C:\Users\spullabhotla\Desktop\TestToRemoveDuplicates.txt
+";
 open (RESULT_FILE,">$result_file");
 
foreach my $line (sort {$no_dupes{$a} <=> $no_dupes{$b}} keys %no_dupe
+s){
 print RESULT_FILE "$line\n";
 }

[download]
When I executed the query the console threw an error : Can't locate File/IO.pm in @INC (@INC contains: C:/Perl64/site/lib C:/Perl64/lib .) at C:\Perl64\Perl Programs\Learning\remit_dup.pl line 5. BEGIN failed--compilation aborted at C:\Perl64\Perl Programs\Learning\remit_dup. pl line 5. Please let me know if I need to change anything in the code.

In reply to Remove Duplicates from one text file and print them in another text file. by bedohave9

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.