in reply to Problem running shell command from Perl

1) Windows has a registry switch that turns on support for 8+3 short name access on the file system. Short names are not supported by default on NTFS file systems on Windows 11. This means you'll have to tweak your system a little bit... https://guyrleech.wordpress.com/2014/04/15/ntfs-8-3-short-names-solving-the-issues/

2) The path must be separated by backslash on Windows OS, not forward slashes. If you are using the builtin open() function in Perl, it will accept a path that includes forward slashes, but if you provide that path to a 3rd-party program or command line utility, chances are it won't recognize it as valid and will complain.

3) Why do you use this HTML-style syntax in the command line to quote Chinese characters? When you type &#number; then the terminal will not translate that into a UTF character but will treat it as literal characters. So, it will look for a file that literally contains & # 3 2 7 6 9 ; rather than the Chinese character that you want to quote.

4) The Documents, Music, Pictures, and Downloads folder on Windows are special folders, and the actual name of the directory may be different than what appears in Windows. This has given me much confusion at one time, because I was looking at the same directory tree from Windows and then from Linux, and I saw two different things. From Linux, I saw that each directory name was preceded by the word "My" but looking at it from Windows, I did not see the word "My" in front of each directory. So, if you still get the file not found error, maybe try typing "My Music" rather than just "Music."

5) Sorry, I don't know how to do this in Perl, but I do know how I would do it in JavaScript, so I wrote a Perl script that produces a JavaScript code that renames files... This function is called WinRenameFile() and it has a unique syntax. You may use forward slashes, but it requires that you put the UTF character codes enclosed within { } brackets. So, you would use it like this: WinRenameFile("C:/Users/James/Music/{22283}{35486}{25079}{24565}{32769}{27468} Vol. 2", "|Vol2~00A"); Before you do this, I would double check the directory contents though using the WinReadDir() function as shown below:

#!/usr/bin/perl use strict; use warnings; # # This program will print all files and sub-directories within $WORKDI +R # my $WORKDIR = "C:\\Users\\James\\Music"; my @FILELIST = WinReadDir($WORKDIR, 1, 'T|F'); foreach (@FILELIST) { my ($ATTR, $FULLNAME) = split(/\|/, $_); if ($ATTR & 16) { print "\n<D> $FULLNAME"; } # Dir else { print "\n<F> $FULLNAME"; } # File } #print "\n\nSTATUS = ", WinRenameFile("C:/Users/James/Music/{22283}{35 +486}{25079}{24565}{32769}{27468} Vol. 2", "|Vol2~00A"), "\n\n"; exit; ################################################## # File | v2023.1.4 # This function is a customizable readdir() function # for Windows. It returns the directory contents as # an array using a tiny JavaScript program to # collect the data. [Tested with TinyPerl 5.8 on WinXP.] # # Usage: LIST = WinReadDir(PATH, [SUBDIR, [ITEMS, [MAX, [REGEX]]]]) # # NOTE: This function only works in Windows XP or higher! # In Linux and other operating systems, an # empty list will be returned. # # The easiest way to use the function is to simply # provide a path as the first argument: # # my @LIST = WinReadDir('C:\\HOME\\PERL'); # # This will return something like this: # # 00000000000 1672793716 More\ # 00000000002 1656954558 Myfile{1758}.txt # 00000000241 1574651186 SPEAK.VBS # 00000001663 1607229792 Numbers.pl # 00000002456 1670070972 cut.pl # 00000059994 1672797060 filelib.pl # 00000361490 1669902950 lib.pl # # The first 12 digits are the file's size. The next 10 digits # are the file's last modified date given in seconds. # If a file name ends with backslash "\" character, then # that means it's a directory name. # # Each line is a separate array element. # If a name contains unicode characters, the special # characters will appear as a number between brackets. # For example: Myfile{1758}.txt # If you press ALT + 1758, it produces a little star icon. # And that's the code that appears in the file name there. # # If a file or directory name contains the '{' character, # then it will appear as '{123}' Also, if you need to refer # to a directory that contains the '{' then again, # you would do this: # # my @LIST = WinReadDir('C:\\TEMP\\x{123}45}'); # # As a result, the function will look in the directory C:\TEMP\x{45} # # The PATH may also contain special characters: # # my @LIST = WinReadDir('C:\\TEMP\\MyWeirdFolder{9700}\\data', 1); # # The second argument is either 1 or 0. Default is zero. # One means that sub-directories will be scanned as well. # In that case, the output will look something like this: # # 00000000000 1661404204 IMGS\ # 00000000000 1659249130 DOCS\ # 00000014010 1642132972 DOCS\0023.bmp # 00000014060 1115495874 DOCS\index.html # 00000015838 1667767236 rr.bmp # 00000015866 1667184404 MANDEL3.BMP # 00000016730 1141128000 FeatherTexture.bmp # 00000017062 1141128000 IMGS\Coffee Bean.bmp # 00000018938 1666659612 TEMP.jif # # Notice that directories' size is always zero. # Also notice that the list is sorted, and whatever comes # first will determine how the list is sorted. You can # change this order using the 3rd argument: # # my @LIST = WinReadDir('C:\\TEMP', 1, 'N S M'); # # The string 'N S M' will substitute the Name of the file # first, then the Size and finally the Modified date all # separated by spaces. You may use a different separator. # I use space just because it makes it easier to read, but # you could use the '|' character which then would allow # you to split the items using the split() function: # # my @LIST = WinReadDir('C:\\TEMP', 1, 'N|S|M'); # foreach (@LIST) # { # my @ITEM = split(/\|/, $_); # # $ITEM[0] ---> NAME OF FILE # # $ITEM[1] ---> FILE SIZE # # $ITEM[2] ---> MODIFIED DATE # } # # There are more values available. For example, if you want # the full name of the file with path, then use letter 'F': # # my @LIST = WinReadDir('C:\\TEMP', 1, 'S**F'); # # This will produce a list which starts with the file size, # followed by the long file name. It will look something like this: # # 000000000000**C:\TEMP\MyWeirdFolder{931}{931}\ # 000000000000**C:\TEMP\x{123}45}\ # 000000000000**C:\TEMP\x{123}45}\test.txt # 000000000002**C:\TEMP\testing{931}.txt # # These are more values that you can use: # # S = insert file size # N = insert file name # M = insert file last modified date # C = insert file date of creation # A = insert file last accessed date # H = insert file's short name (8+3 format) # F = insert file's full name with path # T = insert file's attributes # # You can create your own customized list using a # combinations of the above letters. # # The 4th argument allows you to limit the directory listing # to only X items. For example, here we request only the # first 10 files in the directory list. And we want the file # attribute first, then date of creation, and the full name: # # my @L = WinReadDir("C:\\WINDOWS", 0, 'T|C|F', 10); # # Returns the following list: # # 0016|1665934687|C:\WINDOWS\Config\ # 0016|1665934687|C:\WINDOWS\Cursors\ # 0016|1665934687|C:\WINDOWS\Help\ # 0016|1665934687|C:\WINDOWS\Media\ # 0016|1665934687|C:\WINDOWS\msagent\ # 0016|1665934687|C:\WINDOWS\repair\ # 0016|1665934687|C:\WINDOWS\system32\ # 0016|1665934687|C:\WINDOWS\system\ # 0018|1665934687|C:\WINDOWS\inf\ # 0021|1665934687|C:\WINDOWS\Fonts\ # # The attribute is a 12-bit integer. # The meaning of the bits is described here: # # 0 = Normal file # 1 = Read-only file # 2 = Hidden file # 4 = System file # 8 = Disk drive volume label (Not a real file) # 16 = Directory # 32 = Archive (most files) # 1024 = Link or shortcut # 2048 = Compressed file # # The 5th argument allows you to filter the results using a # regex enclosed as a string. Now, keep in mind, we are not # using Perl's regex engine. This is nowhere near as # sophisticated, but it's better than nothing. Here, for # example, we search for all executable files in the # Windows directory: # # my @L = WinReadDir("C:\\WINDOWS", 0, 'S bytes, name: F', 10, '/ex +e/i'); # # 000000010752 bytes, name: C:\WINDOWS\hh.exe # 000000015360 bytes, name: C:\WINDOWS\TASKMAN.EXE # 000000025600 bytes, name: C:\WINDOWS\twunk_32.exe # 000000049680 bytes, name: C:\WINDOWS\twunk_16.exe # 000000069120 bytes, name: C:\WINDOWS\NOTEPAD.EXE # 000000069632 bytes, name: C:\WINDOWS\ALCMTR.EXE # 000000086016 bytes, name: C:\WINDOWS\SOUNDMAN.EXE # 000000146432 bytes, name: C:\WINDOWS\regedit.exe # 000000256192 bytes, name: C:\WINDOWS\winhelp.exe # 000000283648 bytes, name: C:\WINDOWS\winhlp32.exe # # In the next example, we want to find all files that have # some special characters in their file name: # # my @L = WinReadDir("C:\\TEMP", 0, 'S M T N', 0, '/[{]+/i'); # # So, we get this list: # # 000000000000 1672896164 0016 MyWeirdFolder{931}{931}\ # 000000000000 1672896244 0032 testing{931}.txt # 000000000000 1672896984 0016 x{123}45}\ # # Usage: LIST = WinReadDir(PATH, [SUBDIR, [PATTRN, [MAX, [REGEX]]]]) # sub WinReadDir { my @DIR; $^O =~ m/MSWIN/i or return @DIR; my $PATH = defined $_[0] ? $_[0] : 'C:\\'; $PATH =~ tr|\/|\\|; # Convert / to \ $PATH =~ tr|\\||s; # Remove duplicate backslash. $PATH =~ s/\\/\\\\/g; # Now double each backslash. my $RET = defined $_[2] ? $_[2] : ''; length($RET) or $RET = 'S M N'; $RET =~ tr|'\r\n\\||d; # Filter out unsafe characters my $START = -1; # Start of separator string my $J = ''; # JavaScript code will go here my @f = ('toASCII(n.slice(PATHLEN))+d', 'fSize(s)', 'toASCII(n)+d', 'fDate(f.DateCreated)', 'fDate(f.DateLastModified)', 'fDate(f.DateLastAccessed)', 'fAttr(f)', 'f.ShortName'); for (my $i = 0; $i < length($RET); $i++) { my $c = index('NSFCMATH', substr($RET, $i, 1)); if ($c >= 0) { if ($START >= 0) { $J .= "'" . substr($RET, $START, $i - $START) . "',"; } $J .= "$f[$c],"; $START = -1; } elsif ($START < 0) { $START = $i; } } if ($START < 0) { $J = substr($J, 0, length($J) - 1); } else { $J .= "'" . substr($RET, $START) . "'"; } undef $RET; # Okay, at this point, $J should contain a list of properties # we want to save from each directory and file. These are things # we just plucked out of @f. For example, to record the file size, # $RET had to include the letter 'S' and when we see the letter S, # we insert "fSize(s)," into $J. This list in $J will then become # part of the JavaScript code. When the JS script runs, it creates # a list, joins the items and pushes the string into an array. my $RECURSIVE = defined $_[1] && $_[1] ? 'DIR(FullName);' : ''; my $MAX = defined $_[3] ? $_[3] : 0; $MAX =~ tr|0-9||cd; # Remove everything except numbers $MAX = ($MAX) ? "if(OUTPUT.length>=$MAX)return;" : ''; my $REGEX = defined $_[4] ? $_[4] : ''; # If the regex match is not true, then we continue reading # the directory, otherwise we add the file to our list. # The Regex only tests the name of the file, not its path. # So, if the path contains the pattern we're looking for, # we won't see that. # If $REGEX is not provided, then it won't become part of the code. if (length($REGEX)) { # Sorry, we need to remove forward slashes and backslashes # among other things to prevent code injection vulnerability: $REGEX =~ tr|\/\\'"<>\r\n||d; $REGEX = "NameOnly=toASCII(FullName+'').split(BS).pop();if(!(/$REG +EX/.test(NameOnly)))continue;"; } my $JSCODE = "PATH=CNV('$PATH');OUTPUT=[];BS='\\\\';PATHLEN=PATH.len +gth+((PATH.slice(-1)==BS)?0:1);try{FSO=new ActiveXObject('Scripting.F +ileSystemObject');DIR(PATH);WScript.StdOut.WriteLine(OUTPUT.sort().jo +in('\\n'));}catch(e){}function PACK(d,n){$MAX var f=d?FSO.GetFolder(n +):FSO.GetFile(n);var s=d?0:f.Size;n+='';OUTPUT.push([$J].join(''));}f +unction CNV(s){var i,P;s=s.split('{');for(i=0;i<s.length;i++){P=s[i]. +indexOf('}');if(P>0)s[i]=String.fromCharCode(s[i].substr(0,P)&0xffff) ++s[i].slice(P+1);}return s.join('');}function DIR(p){var F=FSO.GetFol +der(p),FC,File,FullName;for(FC=new Enumerator(F.SubFolders);!FC.atEnd +();FC.moveNext()){FullName=FC.item();Folder=FSO.GetFolder(FullName);$ +REGEX PACK(BS,FullName);$RECURSIVE}for(FC=new Enumerator(F.files);!FC +.atEnd();FC.moveNext()){FullName=FC.item();$REGEX PACK('',FullName);} +}function toASCII(s){var i,T=[];s+='';for(i=0;i<s.length;i++){c=s.cha +rCodeAt(i);T.push(((c<32&&c!=10&&c!=13)||c>126||c==123)?'{'+c+'}':s.c +harAt(i));}return T.join('');}function fSize(s){return('000000000000' ++s).slice(-12);}function fDate(d){return('0000000000'+(d*1)).slice(-1 +3).substr(0,10);}function fAttr(f){return('0000'+f.Attributes).slice( +-4);}"; mkdir "C:\\TEMP"; my $JSFILE = "C:\\TEMP\\GETDIR.JS"; open(my $FILE, ">$JSFILE") or return @DIR; binmode $FILE; print $FILE $JSCODE; close $FILE; if (-s $JSFILE != length($JSCODE)) { return @DIR; } @DIR = split(/\n/, `CSCRIPT.EXE //NOLOGO $JSFILE`); unlink $JSFILE; return @DIR; } ################################################## # File | v2023.1.3 # This function renames a file whose name contains # special unicode characters. # # Usage: STATUS = WinRenameFile(FULLPATH, NEWNAME, [FORCE]) # # Unicode characters must be placed # between {} brackets in decimal format. # For example: {9674} is the representation of a # little diamond shaped character that you can # replicate by pressing ALT + 9674. # # So, if you have a file called "Myfile{9674}.txt" # and you want to rename it to "Myfile.txt" then simply do: # # WinRenameFile('C:\\HOME\\Myfile{9674}.txt', 'Myfile.txt'); # # This will rename the file. If you want to make sure that # the file gets renamed even if the new name is already taken, # then use 1 for the third argument: # # WinRenameFile('C:\\Users\\Zsolt\\Desktop\\Myfile{9674}.txt', 'Myfi +le.txt', 1); # # And if the new file name exists AND happens to be read-only, # the file will not be renamed. However, if you specify 2 for the # third argument, then the read-only "Myfile.txt" will be deleted # first, and then the file will be renamed anyway: # # WinRenameFile('C:\\HOME\\Myfile{9674}.txt', 'My_Cool_file.txt', 2) +; # # This can be used to remove unicode letters to make # files accessible to simple command-line applications. # # You may use normal forward slash in place of backslash. # It makes things a bit clearer: # # WinRenameFile('C:/HOME/Myfile{9674}.txt', 'My_Cool_file.txt'); # # If the second name starts with a "|" character then it will # change the 8+3 short name of the file if short names are # supported by the system. # # WinRenameFile('C:/HOME/Myfile{9674}.txt', '|MYFILE.TXT'); # # You must not type any slashes in the second name. # The new name must only contain a file name and extension. # If you want to move the file to another directory or another # drive, you should use the builtin rename() function. # # This function returns non-zero on success or # zero if the file could not be renamed. # # NOTE: This function only works in Windows XP or higher! # In Linux and other operating systems, no change will # take place and the function always returns zero. # # Usage: STATUS = WinRenameFile(FULLPATH, NEWNAME, [FORCE]) # sub WinRenameFile { $^O =~ m/MSWIN/i or return 0; defined $_[0] && defined $_[1] or return 0; my ($OLD, $NEW) = @_; my $FORCE = defined $_[2] ? $_[2] : 0; $OLD =~ tr|\x00-\x1F\"$\|<>||d; # Remove illegal characters $OLD =~ tr|\/|\\|; # Convert / to \ $OLD =~ tr|\\||s; # Remove duplicate backslash. $OLD =~ s/\\/\\\\/g; # Now double each backslash. my $NAME = ($NEW =~ m/\|/) ? 'ShortName' : 'Name'; $NEW =~ tr|\x00-\x1F\"$\|<>||d; # Remove illegal characters $NEW =~ tr|\\|\/|; # Convert \ to / if (index($NEW, '/') >= 0) { return 0; } length($OLD) or return 0; length($NEW) or return 0; my $JSCODE = "FORCE=$FORCE;OLD=CNV('$OLD');NEW=CNV('$NEW');try{FSO=n +ew ActiveXObject('Scripting.FileSystemObject');if(!FSO.FileExists(OLD +)){BYE(0);}if(FORCE){FULL=NEW;if(NEW.indexOf('\\\\')<0){P=OLD.lastInd +exOf('\\\\');if(P>=0)FULL=OLD.substr(0,P+1)+NEW;}if(FORCE==2)FSO.Dele +teFile(FULL,1);else FSO.DeleteFile(FULL);}}catch(e){}try{F=FSO.GetFil +e(OLD);F.$NAME=NEW;BYE(1);}catch(e){BYE(0);}function BYE(x){WScript.Q +uit(x);}function CNV(s){var i,P;s=s.split('{');for(i=0;i<s.length;i++ +){P=s[i].indexOf('}');if(P>0)s[i]=String.fromCharCode(s[i].substr(0,P +)&0xffff)+s[i].slice(P+1);}return s.join('');}"; # print $JSCODE; exit; # FOR DEBUGGING ONLY mkdir "C:\\TEMP"; my $JSFILE = "C:\\TEMP\\RENAMER.JS"; open(my $FILE, ">$JSFILE") or return 0; binmode $FILE; print $FILE $JSCODE; close $FILE; if (-s $JSFILE != length($JSCODE)) { return 0; } my $STATUS = system("CSCRIPT.EXE //NOLOGO $JSFILE"); unlink $JSFILE; return $STATUS; } ##################################################

Now here comes the downvoting crowd. They have nothing smart to say but downvote every post or comment I make. These people actually ruin Perl's reputation. A programming language is only as good as its community, and sadly Perl's community has become kind of toxic. The goal is no longer to help each other but to bully people. Not very nice. Its kind of a childish behavior... The ones I am writing about know who they are.

Replies are listed 'Best First'.
Re^2: Problem running shell command from Perl
by marto (Cardinal) on Jun 20, 2023 at 14:03 UTC

    "They have nothing smart to say but downvote every post or comment I make. These people actually ruin Perl's reputation. A programming language is only as good as its community, and sadly Perl's community has become kind of toxic. The goal is no longer to help each other but to bully people. Not very nice. Its kind of a childish behavior"

    You get lots of feedback on your posts, you have a tendency to ignore, argue against proven failures or better ways of achieving things. Take a step back and look at your previous comments, recently Re: game programming and the responses you got either outright disproving what you wrote (despite the assertion "everything I have said is the truth."), or commenting on the helpfulness or correctness of your post. Like that thread, here someone (unsurprisingly) is trying to achieve something using perl, you provide some JavaScript. Perhaps you should look objectively at your own behaviour before calling out others. Traits of toxic people

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re^2: Problem running shell command from Perl
by CrashBlossom (Beadle) on Jun 20, 2023 at 14:48 UTC
    Thanks for your response.

    Let me give a bit more detail about that I'm trying to do. I have an app that allows the user to browse through directories and select files whose names are then passed to various other applications. Some of these apps can't handle filenames that contain unicode (non 7-bit ascii) characters). I want to modify my app so that if the user selects a file or directory containing unicode characters, my app can create the 8dot3 name on the fly. I can then run the filename through Win32::GetShortPathName to get a name consisting entirely of 7-bit ascii characters.

    To address the points mentioned in your post:

    1. 8dot3 names are already enabled on my system

    2. understood

    3.The perlmonks website converted the chinese characters to the notation that ended up in the post, apparently due to the fact that the filename was contained in a "<code>" section. Here's the real filename:

    C:\Users\James\Music\國語懷念老歌 Vol. 2

    So if I paste the following into a windows command shell

    fsutil file setShortName "C:\Users\James\Music\國語懷念老歌 Vol. 2" "Vol~002"

    the 8dot3 name is created as desired. Now I need to find a way to do this in my perl program.

    4. not a problem in my case

    5. I need to preserve the original file names, so renaming is not an option.

      I have discovered that if the original filename does not contain unicode chars (ie all 7-bit ascii), my program can create an 8dot3 name. So the following works:

      my $trans2 = `fsutil file setShortName \"C:\\Users\\James\\Music\\thewarrior16\" \"THEW~001\"`;

      But if the filename contains unicode, the fsutil command fails. So the following does NOT work:

      my $trans = `fsutil file setShortName \"C:\\Users\\James\\Music\\國語懷念老歌 Vol. 2\" \"Vol2~0UG\"`;

      Is perl somehow mangling the unicode chars before it passes them to fsutil? There's got to be a simple solution to this. But then, considering what a mess perl's handling of unicode is, maybe not ...

      Any more ideas?

        I am no expert, but here is what I would try next in this situation: I would open Notepad and create a BAT file. I would copy and paste whatever command you said works. Then I would save this as TEST.BAT and make sure to select Unicode format after you click on Save As. Okay. Now run the program. Does it still work? It should. But then the next step is to see if you can generate the same BAT file from Perl. Run it and see what happens. When you take the $trans string and save it to a text file in Unicode file format, the output file of your Perl script should match exactly the the file you saved with Notepad. You will be able to compare the content using a Hex Editor to see if the bytes match. They should. Because if the file generated by Notepad works and the file generated by Perl doesn't work, then you know for sure that Perl does something to the string before saving it to a file. That's what I am suspecting, but I don't know. So, I would try to save it to a file first from Perl to be able to catch exactly how and where things go wrong.

        Edit: Let's say that the Notepad version of BAT file works and the Perl version BAT file works also and they match. But then when you put the command between backticks, it fails. That would tell us that when perl sends the string to be executed by the shell, something happens to the string and it changes from Unicode to ASCII. Then I don't know what other option you have. But maybe try system("something > tempfile.txt"); and see what that does.

Re^2: Problem running shell command from Perl
by bliako (Abbot) on Jun 20, 2023 at 07:29 UTC

    Well, I, for one, have upvoted your post because thanks to it I now know that MS-windows offers the CSCRIPT.EXE command which "Starts a script to run in a command-line environment." (https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/cscript). Just that sentence confirms to me that MS apart from incompetent developers also sports illiterate copy editors. Quite funny is the /nologo flag and that the maximum runtime is 9 hours (see /t:<seconds>) which is very wise since MS OSes need a reboot every couple of hours.

    Apropos the content of your script, is this OK? $OLD =~ tr|\\||s;    # Remove duplicate backslash. .

    edit:false alarm for $OLD, sorry I missed the /s modifier

    bw, bliako

      Asks for Perl, gets JavaScript, bad use of open. MS OS do not need a reboot every couple of hours.
      A reply falls below the community's threshold of quality. You may see it by logging in.