jimbus has asked for the wisdom of the Perl Monks concerning the following question:

Alright, here's another stupid one

When I use SMB to DL a CSV file there is something funky happening to the lines in the file. If I move the files with ftp they are fine (we had this discussion before: the source machine doesn't have an FTP server, but is mounted to our laptops, most people's laptops don't have an FTP server or perl, so I need to get the files from the source machine to the processing machine (a solaris 8 machine). I can ftp them using a client on the laptop then open telnet to run the script or use telnet to run the script and have the script do it with SMB).

Reading all the lines and printing them as such:

  open(FILE, "+<", $filename)
       or die ($node_name."Could not open file: $filename");

  @csvArray = <FILE>;

  foreach $csvLine (@csvArray)
  {
    if ($csvLine =~ /AMA/)
    {
      $csvHash{"header"} = $csvLine;
      next;
    }
    @fields = split /,/, $csvLine;
    @tmp = split /\./, @fields[2];
    chomp($csvLine);
    $csvHash{@tmp[0]} = $csvLine;
  }

  @csvArray = sort values(%csvHash);
foreach $mem (@csvArray)
{
  print "LINE:$mem<\n";
}
exit;

[download]

I get the results for FTP:

LINE:9/8/2005 0:00,FRDS,CDR-827705_9_08.FCDR_1.6718.FRDS.11328,1028217
+0,5020,40018,40018,0,0,0<
LINE:9/8/2005 0:00,FRDS,CDR-827805_9_08.FCDR_1.6718.FRDS.11329,8837858
+,4315,34243,34243,0,0,0<
LINE:9/8/2005 0:00,FRDS,CDR-827905_9_08.FCDR_1.6718.FRDS.11330,7652257
+,3736,29711,29711,0,0,0<
LINE:9/8/2005 0:00,FRDS,CDR-828005_9_08.FCDR_1.6718.FRDS.11331,5346713
+,2610,20778,20778,0,0,0<

[download]

This is what I get for SMB (note the print is "LINE:$mem<": 

<INE:9/8/2005 0:00,FRDS,CDR-827705_9_08.FCDR_1.6718.FRDS.11328,1028217
+0,5020,40018,40018,0,0,0
<INE:9/8/2005 0:00,FRDS,CDR-827805_9_08.FCDR_1.6718.FRDS.11329,8837858
+,4315,34243,34243,0,0,0
<INE:9/8/2005 0:00,FRDS,CDR-827905_9_08.FCDR_1.6718.FRDS.11330,7652257
+,3736,29711,29711,0,0,0
<INE:9/8/2005 0:00,FRDS,CDR-828005_9_08.FCDR_1.6718.FRDS.11331,5346713
+,2610,20778,20778,0,0,0

[download]

This is how I'm making the SMB call:

`smbclient //oh-youn-f2/engineering -W Dobson -U $user_login%$user_pas
+swd -c "cd Common/AMA_Reconcile/AMA_Transferred/;get $infile
incoming/$infile"`;

[download]

I'm sure its something like a chomp or something, but I can't figure it out

thanks

Jimbus

Never moon a werewolf!

Replies are listed 'Best First'.
Re: Text strangeness after using SMB to get a file
by Paladin (Vicar) on Sep 09, 2005 at 21:24 UTC
    Since you mention SMB, I am assuming the files come from a Windows machine of some sort. Chances are that the files have DOS line endings, but your script is running on Solaris, which is assuming *nix line endings. Try running the file through a dos2unix utility to change the line endings first.
[reply]

      Or you could just use:

      perl -i.bak -pe 's/\r\n/\n/' *

      Walking the road to enlightenment... I found a penguin and a camel on the way.....
      Fancy a yourname@perl.me.uk? Just ask!!!
[reply]
[d/l]

        So instead of:

        chomp($csvLine);

        I could do:

        $csvLine =~ s/\r\n//;

        ?

        Never moon a werewolf!
[reply]
[d/l]
[select]
Re: Text strangeness after using SMB to get a file
by AReed (Pilgrim) on Sep 09, 2005 at 23:04 UTC
    Paladin has already supplied the solution, but here's the long, drawn out explanation of the symptom that you saw.

    On Windows, line-endings are two characters: \r\n. On Unix line-endings are a single character: \n. When you run your script on Solaris, chomp removes a trailing \n, but leaves the (now trailing) \r (carriage return).

    So, when you print "LINE:$mem<\n", the last character in $mem is a carriage return, which moves the cursor back to the lefthand edge of the screen on the same line before printing the "<" and then the newline that you explicitly asked for.

    This may have been obvious to you after reading Paladin's response, but this trips me up occassionally too and typing this helps me to remember...

[reply]
Re: Text strangeness after using SMB to get a file
by InfiniteSilence (Curate) on Sep 09, 2005 at 22:02 UTC
    I think Paladin is on point with the answer but I have another inquiry: why did you decide to use SMB in the first place? Here is what you said about your solution:

  • The source CSV is on laptops
  • You have a Solaris server that will do the processing
  • You needed to get the files from the laptops to the server

    Why wouldn't you just put a Webserver on the Solaris box and use a CGI Perl script to upload the file and then process it? I mean, from your description your code is going to use SMB to push the file to a server and then connect with Telnet to run a script? What happens when something goes wrong? Will your laptop users know what to do?

    Celebrate Intellectual Diversity

[reply]

      the CSVs have to be modified and put back were they come from, so its a two way process.

      Never moon a werewolf!
[reply]

Back to Seekers of Perl Wisdom