Rename Windows files with Unicode chars

mnooning has asked for the wisdom of the Perl Monks concerning the following question:

I get files in Windows 7 that have wide chars. I need to rename them after stripping the non-ascii characters from them. Opendir and then readdir (in a while clause) does not work because readdir does bytes!

Below is code that I also tried.

# File name = a1.pl
# Script to get rid of wide chars and non-ascii chars
# in a Windows 7 file name.
# This does not work.
# In the Windows 7 file explorer, the file name shows as
#             "z &#8206;ay&#8206;  &#8206;Pow&#8206;.mp4"
# Note: There is at least one embedded wide char in the above
#       pasted file name.
#       The Windows 7 command promt shows the file name
#       as
#             "z ?ay?  ?Pow?.mp4".

use 5.14.2;

# From: how to read unicode filename
# http://www.perlmonks.org/?node_id=536223
open fList, '-|:encoding(UTF-16LE)', 'cmd /U /C dir /W';

# Note: I tried to opendir and readdir.  I got the shortened
#        8.3 character file name whenever a wide character
#        was in the file name.  I could not rename.

foreach (<fList>) {
    utf8::encode($_);
    my $orig_name = $_;
    my $new_name = $_;
    if ($new_name =~ m/.mp4/i) {
    print " 1 orig_name is \"$orig_name\"\n";

    $new_name =~ s![^[:ascii:]]!!ig;
        print " 2 new name  is \"$new_name\"\n";
    rename "$orig_name", "$new_name"; # Does not work
    }
    
}

__END__
In the results below, note that the end double quotes are not
at the end of the file name line!  That should not be!

>a1.pl
 1 orig_name is "z &#915;اؤay&#915;اؤ  &#915;اؤPow&#915;اؤ.mp4
"
 2 new name  is "z ay  Pow.mp4
"
[download]

Comment on Rename Windows files with Unicode chars Download Code

Replies are listed 'Best First'.

Re: Rename Windows files with Unicode chars
by beech (Parson) on Aug 27, 2016 at 01:58 UTC

Win32::Unicode

update: Example

#!/usr/bin/perl --
use strict;
use warnings;
use Win32::Unicode -native;
listDir();
open my($fh), '>:encoding(UTF-8)', qq{I-\x{2665}-Perl} or die $!;
print $fh qq{I-\x{2665}-Perl};
close $fh;
listDir();
rename qq{I-\x{2665}-Perl}, 'I-love-Perl';
listDir();
unlink 'I-love-Perl';
sub listDir {
    my( $dir ) = grep defined, @_, '.';
    my $wdir = Win32::Unicode::Dir->new( );
    
    $wdir->open($dir) or die $!;
    
    for ($wdir->fetch) {
        next if /^\.{1,2}$/;

        my $full_path = "$dir/$_";
        if (file_type('f', $full_path)) {
            print "f $_\n";
        }
        elsif (file_type('d', $full_path)){
            print "d $_\n";
        }
    }
    $wdir->close or die $!;
    print "\n####\n\n";
}
__END__
[download]

$ chcp 65001
Active code page: 65001

$ perl win32-unicode-native-to-ascii.pl
f win32-unicode-native-to-ascii.pl

####

f I-♥-Perl
f win32-unicode-native-to-ascii.pl

####

f I-love-Perl
f win32-unicode-native-to-ascii.pl

####


$

[reply]
[d/l]

Re^2: Rename Windows files with Unicode chars

by mnooning (Beadle) on Aug 27, 2016 at 03:35 UTC

The line

is cheating. You were able to type in the name of the file that was to be renamed because you already knew what the original file name was. The problem is that I will never have such knowledge beforehand.

Can you show code that will read a file name in the given directory, complete with the file's non-ascii characters, save the original file name in some variable, then strip the non-ascii from the said file name, then rename the file (using the original, saved file name) to the new all-ascii file name?

[reply]

Re^3: Rename Windows files with Unicode chars

by beech (Parson) on Aug 27, 2016 at 03:44 UTC

Hi,

Did you forget about sub listDir { ? How does listDir cheat at reading unicode filenames?

[reply]
[d/l]

Re^4: Rename Windows files with Unicode chars

by mnooning (Beadle) on Aug 27, 2016 at 20:23 UTC

Re^2: Rename Windows files with Unicode chars

by Anonymous Monk on Sep 01, 2016 at 17:08 UTC

[reply]

Re^3: Rename Windows files with Unicode chars

by nikosv (Deacon) on Sep 02, 2016 at 04:32 UTC

use Win32::Console;
Win32::Console::OutputCP( 65001 );
use Devel::Peek;
use Win32::OLE qw(in);
binmode(STDOUT, ":utf8");

Win32::OLE->Option(CP => 
                   Win32::OLE::CP_UTF8);
$obj = Win32::OLE->
      new('Scripting.FileSystemObject');
$folder = $obj->GetFolder(".");
$collection= $folder->{Files};

foreach $value (in $collection) {
   $filename= %$value->{Name};
   next if ($filename !~ /.rar/i);
   print $filename,"\n";
   Dump $filename,"\n";
}
[download]

[reply]
[d/l]

Re^2: Rename Windows files with Unicode chars

by dpoppi (Initiate) on Jan 18, 2017 at 07:22 UTC

Win32::Unicode does not compile under 5.24 (mswin32), so I cannot use that code.

[reply]

Re^3: Rename Windows files with Unicode chars

by beech (Parson) on Jan 18, 2017 at 07:35 UTC

Hi,

Win32::Unicode does not compile under 5.24 (mswin32),

Are you sure about that?

Looking at CPAN Testers Reports: Report for Win32-Unicode-0.38 all I see is a failing test -- meaning the module did compile and can be installed

so I cannot use that code.

Maybe you can use an older version of perl ?

[reply]

Re^3: Rename Windows files with Unicode chars

by Athanasius (Archbishop) on Jan 18, 2017 at 07:52 UTC

Hello dpoppi, and welcome to the Monastery!

In the thread for a bug report for Win32::Unicode, there is a recommendation to use Win32::LongPath as an alternative. I haven’t had occasion to use either module myself, but I can confirm that Win32::LongPath installs cleanly under Perl v5.24.0 (Strawberry Perl, Windows 8.1, 64-bit).

Hope that helps,

Athanasius <°(((>< contra mundum Iustus alius egestas vitae, eros Piratica,

[reply]
[d/l]


P is for Practical
	PerlMonks