http://qs1969.pair.com?node_id=721156

cheerful has asked for the wisdom of the Perl Monks concerning the following question:

They starts with the byte order market FFFE or FEFF. I tried something like this:

my $fh = new FileHandle("< $file"); if (! $fh) { die "failed to open list file '$file': $!"; } my $marker; if (2 != read($fh, $marker, 2)) { die "Failed to read the first 2 bytes from $file"; } if ($marker eq $UNICODE_FFFE) { binmode($fh, ":encoding(utf8)"); } else { $fh->seek(0, 0); }

But the following read

$line = <$fh>;

still generates a lot of error

print $line will produces letter alternating with space.

The script deals with just ascii text.

1. What's the proper to detect unicode in file?

2. How do I deal with unicode string in regular expression matching?

3. Do I need to convert unicode to non-unicode string to do string operation incl. matching? If so, what's the way to do that?