Issue with Loop when Apostrophe in Field

jalopez453 has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I have a loop that goes through each text file. It works great, but the problem is when there is an apostrophe in my second field the script errors and does not append the detail. I am not sure what is causing this, but I was wondering if someone would be able to help me identify the issue and possibly fix it.

#!/usr/bin/perl -w
use strict;
use warnings;
use Text::ParseWords;

opendir IN, 'Outpatient';
my @in = grep { /\.txt$/ } readdir IN; # read all file names form dir 
+except names started with dot
closedir IN;

for my $in (@in) {
  open IN, '<', "Outpatient/$in" || next;
  open OUT, '>', "TXT/$in" || die "can't open file Formatted/$in";
  while(<IN>) {
    my @f = quotewords ',', 0, $_;
    print OUT join "," => @f;
  }
  close OUT;
  close IN;
}

INPUT FILE:
"""555522""",MEN'S HEALTH DEPARTMENT,"""12/31"""

OUTPUT FILE:
**BLANKS

EXPECTED OUTPUT:
555522,MEN'S HEALTH DEPARTMENT,12/31
[download]

Comment on Issue with Loop when Apostrophe in Field Download Code

Replies are listed 'Best First'.
Re: Issue with Loop when Apostrophe in Field by pryrt (Abbot) on Oct 01, 2020 at 18:53 UTC
The documentation on Text::ParseWords is pretty sparse, but looking at the tests, it appears to me that it treats pairs of single-quotes (apostrophes) like an alternative to pairs of double-quotes (quotes). Since there is only one apostrophe, my guess is that it is not seeing the end-single-quote, and so not returning any list because of invalid data (in its opinion. Oh, apparently I do have Text::ParseWords, so I tried it: C:>perl -MText::ParseWords -le "print for quotewords ',', 0, qq(First, +Second,Third) " First Second Third C:>perl -MText::ParseWords -le "print for quotewords ',', 0, qq(First, +Second 'Line',Third) " First Second Line Third C:>perl -MText::ParseWords -le "print for quotewords ',', 0, qq(First, +Second 'Line',it's not a bug) " C:\usr\local\share>perl -MText::ParseWords -le "print for quotewords ' +,', 0, qq(First,Second 'Line',it's not a bug,it's a feature) " First Second Line its not a bug,its a feature C:> [download] Yep, quotewords recognizes apostrophes/single-quotes as quotes, so you cannot have unpaired apostrophes. addendum: any reason you aren't using Text::CSV_XS for dealing with CSV? /addendum	[reply] [d/l]
Re: Issue with Loop when Apostrophe in Field by jwkrahn (Abbot) on Oct 01, 2020 at 20:02 UTC
`my @in = grep { /\.txt$/ } readdir IN; # read all file names form dir +except names started with dot` [download] Your code does not exclude file names that start with a dot (hidden files). To do that you need this: `my @in = grep { !/^\./ && /\.txt$/ } readdir IN; # read all file names + from dir except names started with dot` [download] `open IN, '<', "Outpatient/$in" \|\| next; open OUT, '>', "TXT/$in" \|\| die "can't open file Formatted/$in";` [download] Your opens will not fail correctly because of the high precedence of the `\|\|` operator. You need to either use the `or` operator (low precedence): `open IN, '<', "Outpatient/$in" or next; open OUT, '>', "TXT/$in" or die "can't open file Formatted/$in";` [download] Or use parentheses: `open( IN, '<', "Outpatient/$in" ) \|\| next; open( OUT, '>', "TXT/$in" ) \|\| die "can't open file Formatted/$in";` [download]	[reply] [d/l] [select]
Re: Issue with Loop when Apostrophe in Field by perlfan (Parson) on Oct 01, 2020 at 18:48 UTC
Idk, man. Text::Parse::parse_line looks ridiculously complex for what you're doing. Can you describe that record format? Looks rather short sighted, but something that can be "parsed" without a fancy module.	[reply]