Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re^5: What is the proper way to read non-ANSI data

by freonpsandoz (Beadle)
on Oct 04, 2015 at 00:39 UTC ( #1143729=note: print w/replies, xml ) Need Help??


in reply to Re^4: What is the proper way to read non-ANSI data
in thread What is the proper way to read non-ANSI data

Upon further investigation, I found that dumptorrent is capable of dumping more than one torrent with a single command. Dumping both a torrent containing a centered dot and torrent containing the EN DASH to a single file does indeed yield a mixed encoding. At this point, I don't know if this is due to different encodings used in different torrent files, a problem with dumptorrent, a problem with the bencode library, or a problem with the Windows APIs. I found that I could read the data correctly by trying to decode raw input as UTF-8 and, if that fails, decoding it as cp1252. This seems to work for all torrent files I have encountered so far:

use strict; use warnings; use Try::Tiny; use Regexp::Common qw /number/; use Encode; my $linecount = 0; open(DATA, qq(dumptorrent.exe "$ARGV[0]"|)); binmode(DATA, ":raw"); binmode(STDOUT, ":encoding(UTF-8)"); foreach my $line (<DATA>) { $line = DecodeRawLine($line); # Discard the 5 lines of output before the list of filenames if (++$linecount > 5) { last if (length($line) == 0); $line =~ s/^ *//; $line =~ s/ +\(${RE{num}{real}}[KMG]\)$//; # e.g. " (8.22M +)" my $sizeindex = rindex($line, ' ') + 1; my $filesize = substr($line, $sizeindex); # Ignore zero-length files in torrents. if (defined $filesize && length($filesize) > 0 && $filesiz +e > 0) { # Remove the extra spaces between the file name and si +ze (my $filekey = $line) =~ s/ +(\d+)$/ $1/; print "$filekey\n"; } } } sub DecodeRawLine { # IMPORTANT! Any arguments in @_ that are needed in 'try' must be +COPIED! my $line = substr($_[0], 0, length($_[0])-2); # remove CR-LF try { $line = Encode::decode('UTF-8', $line, Encode::FB_CROAK); } catch { $line = Encode::decode('cp1252', $line); }; return $line; }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1143729]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2022-08-10 05:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?