Calling Windows Batch from Perl, double-quote appearing out of nowhere

rovf has asked for the wisdom of the Perl Monks concerning the following question:

(I tested this with ActiveState Perl 5.8.8 and 5.12 on Windows 7 (64 bit))

I have the following batch file testing.bat:

echo XXXXXXX  >X_%2.log 2>&1
sleep 5
[download]

Note that this batch file ignores its first parameter and just uses its second. It seems to be important to demonstrate this bug to have a batch file which will be called with 2 parameters.

Further, I have the following Perl program testing.pl:

use strict;
use warnings;
my $cmdline="cmd /c testing.bat AA 1";      
my $child_pid=system(1,$cmdline);  
print "$child_pid\n";
wait;
print "Finished\n";
exit;
[download]

This just calls testing.bat with in background, and terminates when the child also terminated. Running the batch file in background seems to be crucial for this bug (?) to occur. BTW, the sleep in the batch file is not essential to demonstrate the problem, but it shows that the wait indeed waits until the child terminates.

When I run perl testing.pl, I get the following output:

C:\Users\rofischx\playground>perl testing.pl
4864

C:\Users\rofischx\playground>echo XXXXXXX   1>X_1".log 2>&1
The filename, directory name, or volume label syntax is incorrect.
C:\Users\rofischx\playground>sleep 5
Finished
[download]

We see that there occurs suddenly a double quote inside the redirection, which in turn causes the error message. Any idea where this spurious double-quote comes from? Also, could someone try to reproduce this behaviour, either with the same Perl version, or with a newer one, an let me know the outcome?

Also interesting: From my experiments with this problem, the program posted here seems to be the minimal one. Whenever I tried to simplify it further (reducing the number of parameters in the batchfile, for instance), the problem did not show up.

--
Ronald Fischer <ynnor@mm.st>

Comment on Calling Windows Batch from Perl, double-quote appearing out of nowhere Select or Download Code

Replies are listed 'Best First'.
Re: Calling Windows Batch from Perl, double-quote appearing out of nowhere (insanity) by tye (Sage) on Nov 09, 2012 at 15:07 UTC
People expect to be able to run `system("dir .foo")` from Perl. But there is no 'dir' executable so to get that to work, Perl needs to actually run: `cmd /c "dir .foo"` So what Perl ran for you was: `C:\> cmd /c "cmd /c systemQuote.bat AA 1" C:\> echo XXXXXXX 1>X_1".log 2>&1 The filename, directory name, or volume label syntax is incorrect.` [download] Which shows that Microsoft has a hard time dealing with quotes in a consistent or even sane and error-free manner. But I tried this: `C:\> cmd /c "cmd /c systemQuote.bat AA 1 " C:\> echo XXXXXXX 1>X_1.log 2>&1 C:\> sleep 5` [download] And putting a trailing space also worked around the problem when running that from Perl. It used to be that Perl's system() on Windows would try running the command you gave directly first and would only resort to wrapping it in `qq(cmd /c "...")` if that failed. I think that, over the years, enough edge cases showed up that the details of this technique were changed in order to try to get more "expected behavior" in more cases. If you are interested in details, then check out the source code involved, p5git://win32/win32.c.. - tye	[reply] [d/l] [select]
Re^2: Calling Windows Batch from Perl, double-quote appearing out of nowhere (insanity) by rovf (Priest) on Nov 09, 2012 at 15:26 UTC
Interesting point - this indeed explains (and it is also interesting that the quoting problem only shows up when we invoke the background version, but not the "normal" call to system). It used to be that Perl's system() on Windows would trying running the command you gave directly first and would only resort to wrapping it in qq(cmd /c "...") if that failed. I think that, over the years, enough edge cases showed up that the details of this technique were changed in order to try to get more "expected behavior" in more cases. Would this mean that in newer versions of Perl, the "foregroun" system() would show the same problem? I thought of using system() instead of system(1,...) and doing an explicit fork and exect, to avoid the quoting bug, but if newer Perl versions will consistently wrap the argument inside an extra invocation of cmd /c, the problem would reappear if I upgrade to a newer Perl :-( -- Ronald Fischer <ynnor@mm.st>	[reply] [d/l]
Re^3: Calling Windows Batch from Perl, double-quote appearing out of nowhere (insanity) by tye (Sage) on Nov 09, 2012 at 15:36 UTC
Just for y'all's information, I don't know the answer to that. It would be interesting to look into why only system(1,...) wraps things in `cmd /c "..."` in a way that triggers this bug. To figure that out, I'd read the code that I linked to above. I did scan some parts of it earlier but not looking for that quirk (I initially missed your point that the "1," was required). But why not just include a trailing space and then none of that will likely matter? (And include a comment noting that the trailing space is required to work around a bug in: cmd /c "cmd /c ...".) - tye	[reply] [d/l]
Re^4: Calling Windows Batch from Perl, double-quote appearing out of nowhere (insanity) by rovf (Priest) on Nov 09, 2012 at 15:45 UTC
Re^5: Calling Windows Batch from Perl, double-quote appearing out of nowhere (insanity) by Anonymous Monk on Nov 09, 2012 at 15:52 UTC
Re^2: Calling Windows Batch from Perl, double-quote appearing out of nowhere (insanity) by Anonymous Monk on Nov 09, 2012 at 15:29 UTC
FWIW it actually ran `$ perl -V:sh sh='cmd /x /c';` [download]	[reply] [d/l]
Re: Calling Windows Batch from Perl, double-quote appearing out of nowhere by Anonymous Monk on Nov 09, 2012 at 14:24 UTC
well, I get the error if using cmd /c but not calling the batch file directly don't use cmd /c and it will work don't use cmd /c esp since you're using system 1, which you use so you don't have to spawn a program in background through cmd /c "start ..." See also Behind the GUI lives the Shell Windows Command Line Interpreter\|Shell\|DOS Prompt\|Batch Files\|Scripting Csh Programming Considered Harmful How Command Line Parameters Are Parsed http://www.gnu.org/software/bash/manual/bashref.html#Quoting String::ShellQuote/Win32::ShellQuote, Re: system() implementation on Windows (again), Re^3: system() implementation on Windows (again), Everyone quotes command line arguments the wrong way Proc::Background The Windows NT Command Shell exec always invokes the shell? win32, System, exec different behaviour with spaces	[reply]
Re^2: Calling Windows Batch from Perl, double-quote appearing out of nowhere by rovf (Priest) on Nov 09, 2012 at 15:17 UTC
well, I get the error if using cmd /c but not calling the batch file directly don't use cmd /c and it will work Thanks a lot! I could verify that, what you said, is true (though it only helps partially here, because in our application, we receive the commands to be executed from an outside source, and we can't control how they are created - but this is a different matter). However, I don't feel that the explanation> you provided, is really correct to explain the case, for the following reason: A batch file can't be run "by itself" - it always needs a CMD.EXE (or a compatible command interpreter) to run it; so, even if I pass the batchfile name directly, Perl (or Windows?) must, under the hood, spawn cmd.exe somewhere, in order to execute the batch file. Thus I don't think it has to do with whether or not CMD is used (it always is), but somewhat in the way system is implemented, manages to add the incorrect double quote in certain cases... -- Ronald Fischer <ynnor@mm.st>	[reply] [d/l]
Re^3: Calling Windows Batch from Perl, double-quote appearing out of nowhere by Anonymous Monk on Nov 09, 2012 at 15:26 UTC
I can't tell, are you asking me something?	[reply]
Re^4: Calling Windows Batch from Perl, double-quote appearing out of nowhere by rovf (Priest) on Nov 09, 2012 at 15:29 UTC
Re: Calling Windows Batch from Perl, double-quote appearing out of nowhere by karlgoethebier (Abbot) on Nov 09, 2012 at 14:52 UTC
Try (in testing.bat): `echo XXXXXXX > X_%%2.log 2>&1`. Regards, Karl �The Crux of the Biscuit is the Apostrophe�	[reply] [d/l]
Re^2: Calling Windows Batch from Perl, double-quote appearing out of nowhere by rovf (Priest) on Nov 09, 2012 at 15:40 UTC
Indeed, this works too! (big surprise!) UPDATE: Sorry, responded to quickly. No, it doesn't work. The %% is a notion to escape the percent character, so instead of interpolating the 2nd argument, it just inserts a literal percent sign... -- Ronald Fischer <ynnor@mm.st>	[reply] [d/l]
Re^3: Calling Windows Batch from Perl, double-quote appearing out of nowhere by Anonymous Monk on Nov 09, 2012 at 15:49 UTC
not really :) you're using cmd to call cmd and the escaping is botched -- double interpolation	[reply]
Re: Calling Windows Batch from Perl, double-quote appearing out of nowhere by space_monk (Chaplain) on Nov 09, 2012 at 14:31 UTC
Runs fine in cygwin with the following changes: what is the '1' parameter in the system call for? Had to remove first param. Added #!/usr/bin/perl at start Update: 1 has been answered below, but it seems that feature not supported in Perl under cygwin A Monk aims to give answers to those who have none, and to learn from those who know more.	[reply]
Re^2: Calling Windows Batch from Perl, double-quote appearing out of nowhere by rovf (Priest) on Nov 09, 2012 at 15:18 UTC
but it seems that feature not supported in Perl under cygwin The feature isn't supported on Unix/Linux either - I think that's the reason why it is also absent from Cygwin Perl. -- Ronald Fischer <ynnor@mm.st>	[reply] [d/l]
Re^2: Calling Windows Batch from Perl, double-quote appearing out of nowhere by Anonymous Monk on Nov 09, 2012 at 14:37 UTC
its a wart, see perlport#system, although, if you don't know the answer, or the question, might want to pause and consider if you should jump in :)	[reply]
Re^3: Calling Windows Batch from Perl, double-quote appearing out of nowhere by space_monk (Chaplain) on Nov 09, 2012 at 14:54 UTC
Actually I ran the OPs program on my system, and the program failed because of the wart, so it was a legitimate query. My perl version (5.14) is more recent than the OP, so the program should run, warts and all. :-) A Monk aims to give answers to those who have none, and to learn from those who know more.	[reply]
Re^4: Calling Windows Batch from Perl, double-quote appearing out of nowhere by Anonymous Monk on Nov 09, 2012 at 15:00 UTC
Re^5: Calling Windows Batch from Perl, double-quote appearing out of nowhere by space_monk (Chaplain) on Nov 09, 2012 at 15:10 UTC