[SOLVED]: Problem when using chomp on line read from file.

Perl300 has asked for the wisdom of the Perl Monks concerning the following question:

Using perl (v5.24.0) I am reading a file line by line processing each line in while loop.

As I have newlines at the end of each line in the file, I want to remove those newlines from every line I read in. But when I am using chomp, something I didn't expect is happening. I am not able to find why it's happening but I suspect the way I am reading the file might be responsible for it.

#!$HOME/localperl/bin/perl

use strict;
use warnings;
use diagnostics;

my $file = "test1.csv";
open(my $fh, "<", $file) or die "Could not open file '$file' $!\n";
my @local;
print "Before while \@local: @local\n";
while (my $line = <$fh>)
{
    print "Before chomp \$line: $line\n";
    #chomp($line);
    print "After chomp \$line: $line\n";
    push @local, $line;
    print "After push \@local: @local\n";
}
print "After while \@local: @local\n";
close $fh;
[download]

Contents of test1.csv:

12.2(17d)SXB11
5.10 Generic_118855-15
XYZ 7.1
[download]

When I run the code with chomp commented out, I get following output

Before while @local:
Before chomp $line: 12.2(17d)SXB11

After chomp $line: 12.2(17d)SXB11

After push @local: 12.2(17d)SXB11

Before chomp $line: 5.10 Generic_118855-15

After chomp $line: 5.10 Generic_118855-15

After push @local: 12.2(17d)SXB11
 5.10 Generic_118855-15

Before chomp $line: XYZ 7.1

After chomp $line: XYZ 7.1

After push @local: 12.2(17d)SXB11
 5.10 Generic_118855-15
 XYZ 7.1

After while @local: 12.2(17d)SXB11
 5.10 Generic_118855-15
 XYZ 7.1
[download]

Now, as I want to remove the newline from end of each line, when I uncomment chomp, I get:

Before while @local:
Before chomp $line: 12.2(17d)SXB11

After chomp $line: 12.2(17d)SXB11
After push @local: 12.2(17d)SXB11
Before chomp $line: 5.10 Generic_118855-15

After chomp $line: 5.10 Generic_118855-15
 5.10 Generic_118855-15(17d)SXB11
Before chomp $line: XYZ 7.1

After chomp $line: XYZ 7.1
 XYZ 7.1neric_118855-15(17d)SXB11
 XYZ 7.1neric_118855-152(17d)SXB11
[download]

What am I ignoring here that is causing this behavior? I want to add each line as an array element in the @local after removing newlines. Please forgive me if I am missing something very obvious.

Comment on [SOLVED]: Problem when using chomp on line read from file. Select or Download Code

Replies are listed 'Best First'.
Re: Problem when using chomp on line read from file. by choroba (Cardinal) on Feb 26, 2018 at 21:58 UTC
The input file uses MSWin line end, i.e. there's a \r before each \n. Either open the file with :crlf, or remove the \r's manually. ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l]
Re^2: Problem when using chomp on line read from file. by Perl300 (Friar) on Feb 26, 2018 at 22:19 UTC
Thank you so much for your response. I got it fixed now by adding following line before the while loop `local $/ = "\r\n";` Now I get the desired output as: `Before while @local: Before chomp $line: 12.2(17d)SXB11 After chomp $line: 12.2(17d)SXB11 After push @local: 12.2(17d)SXB11 Before chomp $line: 5.10 Generic_118855-15 After chomp $line: 5.10 Generic_118855-15 After push @local: 12.2(17d)SXB11 5.10 Generic_118855-15 Before chomp $line: XYZ 7.1 After chomp $line: XYZ 7.1 After push @local: 12.2(17d)SXB11 5.10 Generic_118855-15 XYZ 7.1 After while @local: 12.2(17d)SXB11 5.10 Generic_118855-15 XYZ 7.1` [download]	[reply] [d/l] [select]
Re: [SOLVED]: Problem when using chomp on line read from file. by johngg (Canon) on Feb 26, 2018 at 23:01 UTC
I see that you have resolved your problem while I was composing this response but I might as well post it anyway in case it is helpful. As you found, chomping MS Windows line terminators on a *nix type system without taking extra measures causes problems. Using split, ord, sprintf and map can be helpful in visualising what is happening to your data, it is much clearer than just printing each line. Your solution is to modify the default line terminator, which is the second item in the following code. Note the difference pre-chomp between that method and the `:crlf` I/O layer in my third item which actually removes the carriage return during the readline leaving just the line feed. use strict; use warnings; use feature qw{ say }; my $winLineTerm = qq{\x0d\x0a}; my( $file, $writeFH, $readFH ); say q{-} x 20; open $writeFH, q{>}, \ $file or die $!; print $writeFH qq{ABC${winLineTerm}DEF$winLineTerm}; close $writeFH or die $!; say q{No measures}; open $readFH, q{<}, \ $file or die $!; while ( <$readFH> ) { say q{Before chomp()}; say for map { sprintf q{ %#02x}, ord } split m{}; say q{After chomp()}; chomp; say for map { sprintf q{ %#02x}, ord } split m{}; } close $readFH or die $!; say q{-} x 20; say q{Change default line terminator}; { local $/ = $winLineTerm; open $readFH, q{<}, \ $file or die $!; while ( <$readFH> ) { say q{Before chomp()}; say for map { sprintf q{ %#02x}, ord } split m{}; say q{After chomp()}; chomp; say for map { sprintf q{ %#02x}, ord } split m{}; } close $readFH or die $!; } say q{-} x 20; say q{open() with :crlf I/O layer}; open $readFH, q{<:crlf}, \ $file or die $!; while ( <$readFH> ) { say q{Before chomp()}; say for map { sprintf q{ %#02x}, ord } split m{}; say q{After chomp()}; chomp; say for map { sprintf q{ %#02x}, ord } split m{}; } close $readFH or die $!; say q{-} x 20; [download] The output. -------------------- No measures Before chomp() 0x41 0x42 0x43 0xd 0xa After chomp() 0x41 0x42 0x43 0xd Before chomp() 0x44 0x45 0x46 0xd 0xa After chomp() 0x44 0x45 0x46 0xd -------------------- Change default line terminator Before chomp() 0x41 0x42 0x43 0xd 0xa After chomp() 0x41 0x42 0x43 Before chomp() 0x44 0x45 0x46 0xd 0xa After chomp() 0x44 0x45 0x46 -------------------- open() with :crlf I/O layer Before chomp() 0x41 0x42 0x43 0xa After chomp() 0x41 0x42 0x43 Before chomp() 0x44 0x45 0x46 0xa After chomp() 0x44 0x45 0x46 -------------------- [download] I hope this is of interest. Cheers, JohnGG	[reply] [d/l] [select]