They starts with the byte order market FFFE or FEFF. I tried something like this:

        my $fh = new FileHandle("< $file");
        if (! $fh) {
        die "failed to open list file '$file': $!";
        }
        my $marker;
        if (2 != read($fh, $marker, 2)) {
        die "Failed to read the first 2 bytes from $file";
        }
        if ($marker eq $UNICODE_FFFE) {
        binmode($fh, ":encoding(utf8)");
        }
        else {
        $fh->seek(0, 0);
        }
[download]

But the following read

$line = <$fh>;
[download]

still generates a lot of error

print $line will produces letter alternating with space.

The script deals with just ascii text.

1. What's the proper to detect unicode in file?

2. How do I deal with unicode string in regular expression matching?

3. Do I need to convert unicode to non-unicode string to do string operation incl. matching? If so, what's the way to do that?