consecutive spaces in piped open

Woodya has asked for the wisdom of the Perl Monks concerning the following question:

Greetings.

I've run into a strange situation with file names containing multiple consecutive spaces passed through a piped open call. The platform I'm testing on (and the target running environment) is Perl 5.6.1 (ActiveState) Win32 build running on a Win2K server. The problem can be demonstrated with the below sample:

#!/usr/bin/perl
use strict;
my $filename = 'This_filename_has_2  2_spaces.txt';
my $tempstr = "echo \"$filename\" |";
printf "Below is the output of \'$tempstr\'\n";
open (IN,$tempstr) or die "Failed to echo $filename!\n";
printf "\t";
printf <IN>;
close (IN);
[download]

The output looks like:

Below is the output of 'echo "This_filename_has_2  2_spaces.txt" |'
        This_filename_has_2 2_spaces.txt
[download]

The output of the first print statement maintains the two spaces in the filename, but the response from the echo has condensed it to a single space. Same thing happens with multiple spaces.

I've tried different quote delimeters, as well as using sysopen (which I couldn't figure out how to use as a pipe in this same way). The escaped double quotes on the filename are needed for real external calls like md5sum.exe, since it's Win32. I'm wondering if this is a quirk of the Win32 environment, or if this happens elsewhere as well. Suggestions?

Comment on consecutive spaces in piped open Select or Download Code

Replies are listed 'Best First'.
Re: consecutive spaces in piped open by BrowserUk (Patriarch) on Dec 25, 2002 at 09:27 UTC
Your code produces your desired output on my machine under NT/AS 5.6.1 using the standard shell. Not sure quite what this shows, except that maybe you are running your code under the auspices of cygwin or 4NT maybe? I which case, you may need to look at the quoting rules for that environment. If not, it could be that they changed the rules for Win2K, which would be no surprise. `c:\test>type temp.pl #! perl -sw use strict; my $filename = 'This_filename_has_2 2_spaces.txt'; my $tempstr = "echo \"$filename\" \|"; printf "Below is the output of \'$tempstr\'\n"; open (IN,$tempstr) or die "Failed to echo $filename!\n"; printf "\t"; printf <IN>; close (IN); c:\test>temp.pl Below is the output of 'echo "This_filename_has_2 2_spaces.txt" \|' "This_filename_has_2 2_spaces.txt" c:\test>` [download] Examine what is said, not who speaks.	[reply] [d/l]
Re: consecutive spaces in piped open by dws (Chancellor) on Dec 25, 2002 at 08:59 UTC
I've run into a strange situation with file names containing multiple consecutive spaces passed through a piped open call. Here's a semi-ugly workaround that might work for you: Rename the file to a sensible temporary name, do the piped open and process the file, then rename it back.	[reply]
Re: consecutive spaces in piped open by pg (Canon) on Dec 25, 2002 at 17:42 UTC
I have windows 98 around, but yet not be able to reproduce your problem, seems that no one around can reproduce it. Why don't you take Perl out of the loop, design a simple test: go to your dos command prompt, and type, echo WithMany Spaces and see whether it makes all the spaces into one, this can narrow down the problem, and tell whether it is with the echo command of your dos/win system, or your version of Perl.	[reply]
Re: consecutive spaces in piped open by traveler (Parson) on Dec 25, 2002 at 16:55 UTC
I think BrowserUK is right. Try embedding quotes in the $filename string: `my $filename = q('This_filename_has_2 2_spaces.txt');` [download] HTH, --traveler	[reply] [d/l]
Re: consecutive spaces in piped open by Woodya (Novice) on Dec 25, 2002 at 18:50 UTC
Thanks for the feedback all! dws: I can't rename the files, 'cause part of the information I'm collecting (in the larger script) is the last modified time. I guess I could copy the file to a temp file with a reasonable name, but that could be trouble with very large files. BrowserUK: Good call! I think it might be hitting cygwin. I'll have to try it on a Win2K box without cygwin tools installed. traveler: I tried embedding the quotes, but it didn't change the result. I think it might be the shell env. pg: It's not just echo, other commands do it too. Any command called straight from the shell works fine, but only when they're called from open does this happen. More testing to come (tomorrow). I'll let you know if I find anything!	[reply]
Re: consecutive spaces in piped open by MarkM (Curate) on Dec 26, 2002 at 07:08 UTC
Make sure you are using ActivePerl Build 633. Several versions before 633 including sub-command invocation problems. I've had mixed experience with the use of Perl to invoke sub-commands that contain other the simplest of commands. Internally, unless Perl decides to create the sub-command process itself, the command is passed to 'cmd /x/c <YOUR_COMMAND_HERE>'. Under certain circumstances, 'cmd.exe' is not bright enough to know that 'cmd /x/c echo "line with double spaces"' is not the same as 'cmd /x/c echo line with double spaces'. Using ActivePerl 633 on WIN XP I am not able to reproduce your particular problem. I do recall having similar issues in the past. If you can't fix the problem by upgrading, another alternative would be to create the process explicitly using Win32::Process, effectively avoiding cmd.exe altogether.	[reply]
Re: consecutive spaces in piped open by Woodya (Novice) on Dec 31, 2002 at 22:37 UTC
Well, after much testing (and confusion with Win32::Process ;) I finally upgraded to build 633 and watched the whole ugly problem disappear. Best I can tell, it looks like MarkM's description of sub-command invocation problems was what I was really seeing. It looks like Perl didn't know when to stop parsing the arguments before passing them to cmd.exe. Thanks all for the help!	[reply]
Re: consecutive spaces in piped open by bart (Canon) on Dec 28, 2002 at 10:15 UTC
`my $filename = 'This_filename_has_2 2_spaces.txt'; my $tempstr = "echo \"$filename\" \|"; printf "Below is the output of \'$tempstr\'\n";` [download] It may look like a filename to you, but there's no filename there. All I can see is an "echo" command with a string. From the result printed by echo, the double spaces are turned into a single space. So it seems pretty clear to me that it's the version of `echo` that you have, that causes this.	[reply] [d/l]