Re: Sorting/Cleansing a Duplicate File

If the file has a .txt extension, I guess it could be a "word document", a file containing words. That name suggest a Microsoft Word document, which probably would make this a bit trickier.

1 Peter 4:10

Comment on Re: Sorting/Cleansing a Duplicate File

Replies are listed 'Best First'.
Re^2: Sorting/Cleansing a Duplicate File by perlron (Pilgrim) on Oct 26, 2014 at 12:13 UTC
Can i have a link to understand how to deal with tricky utf8 issues when working with text in perl ? Do not wait to strike when the iron is hot! Make it hot by striking - WB Yeats	[reply]
UTF-8 encodings issues (Re^3: Sorting/Cleansing a Duplicate File) by soonix (Chancellor) on Oct 26, 2014 at 15:00 UTC
see current thread Problems handling UTF8 ! And removing accents. I have bookmarked Re: UTF-8 for Everything (mostly because of the links in there) In my scripts (mostly Windows console), for data (text) files I often use `open (my $fh, '<:encoding(Windows-1252)', $fname) or die "cannot open +$fname: $!";` [download] and for output intended for the console window I use `binmode STDOUT, ':encoding(cp437)'; # or cp850` [download] (if your data really is UTF-8, then "encoding(utf-8)" should do the right thing Also of interest: perluniintro and perlopentut	[reply] [d/l] [select]