When I parse a csv file, certain records get the following error:
2023/EIQ - QUO character not allowed/105/3/7
The data:
147;lakjfh lkjsfh ehjd;134-324-730 31;291;24.04.2020;15 000,00;severo- +vostocnoe otdelenie № 8645 pao "sberbank rossii";4243972345;347 +636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao sber +bank g. magadan;896868986;98375423895239529987;;96764128476876487264 148;lkjf fdkas fa;123-105-878 17;1;23.04.2020;15 000,00;severo-vostocn +oe otdelenie № 8645 pao "sberbank rossii";4243972345;347636334; +23452347344633423542;severo-vostocnoe otdelenie N8645 pao sberbank g. + magadan;896868986;98375423895239529987;;23236726352762456346 149;sdfg gdsgsdhsds shsddf;104-424-501 02;146;20.04.2020;15 000,00;sev +ero-vostocnoe otdelenie pao "sberbank";4243972345;347636334;234523473 +44633423542;severo-vostocnoe otdelenie N8645 pao sberbank g. magadan; +896868986;98375423895239529987;;23236726352762456346 150;dfgsdfgsdg sdgsdgsdgsdg sdfgdsgs;095-504-250 68;68;17.04.2020;15 0 +00,00;"aziatsko-tihookeanskiy bank" (pao) g. blagovesensk;3473446334; +280101001;23452344637833423542;"aziatsko-tihookeanskiy bank" (pao) g. + blagovesensk;098676868;9837542asdfaas529987;;23236726352762456346 151;sdfgds fdsgd ssdgsd;108-437-022 37;258;23.04.2020;15 000,00;severo +-vostocnoe otdelenie № 8645 pao "sberbank rossii";4243972345;34 +7636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao sbe +rbank g. magadan;896868986;98375423895239529987;;23236726352762456346 152;bgerghe egertge ertgeer;074-073-043 41;128;20.04.2020;15 000,00;se +vero-vostocnoe otdelenie № 8645 pao "sberbank rossii";424397234 +5;347636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao + sberbank g. magadan;896868986;98375423895239529987;;2323672635276245 +6346 153;lskdjfa sflaskjfd aslkdfjaslf;151-533-432 32;33;22.04.2020;15 000, +00;severo-vostocnoe otdelenie № 8645 pao "sberbank rossii";4243 +972345;347636334;23452347344633423542;severo-vostocnoe otdelenie N864 +5 pao sberbank g. magadan;896868986;98375423895239529987;;23236726352 +762456346 154;lasfnf fsdafasdfas afs;134-092-549 45;21;23.04.2020;5 000,00;sever +o-vostocnoe otdelenie № 8645 pao "sberbank rossii";4243972345;3 +47636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao sb +erbank g. magadan;896868986;98375423895239529987;;2323672635276245634 +6 155;asdfasf asdf asdfasfd;110-497-874 55;50;24.04.2020;15 000,00;sever +o-vostocnoe otdelenie № 8645 pao "sberbank rossii";4243972345;3 +47636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao sb +erbank g. magadan;896868986;98375423895239529987;;2323672635276245634 +6 156;sadfasf asdf asdfas;456-978-244 89;117;17.04.2020;15 000,00;severo +-vostocnoe otdelenie № 8645 pao "sberbank rossii";4243972345;34 +7636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao sbe +rbank g. magadan;896868986;98375423895239529987;;23236726352762456346 157;asdfasfwer asfdasfs sadf;139-220-696 59;26;21.04.2020;15 000,00;se +vero-vostocnoe otdelenie № 8645 pao "sberbank rossii";424397234 +5;347636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao + sberbank g. magadan;896868986;98375423895239529987;;2323672635276245 +6346 158;lksj ljlkjlkjl lkjljlk;133-087-587 59;262;23.04.2020;15 000,00;sev +ero-vostocnoe otdelenie № 8645 pao "sberbank rossii";4243972345 +;347636334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao +sberbank g. magadan;896868986;98375423895239529987;;23236726352762456 +346 159;fghd g dfghdhdfgh;141-008-388 12;241;22.04.2020;30 000,00;severo-v +ostocnoe otdelenie № 8645 pao "sberbank rossii";4243972345;3476 +36334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao sberb +ank g. magadan;896868986;98375423895239529987;;23236726352762456346 160;lkkljh kljhk kjh;123-650-136 21;20;20.04.2020;15 000,00;severo-vos +tocnoe otdelenie № 8645 pao "sberbank rossii";4243972345;347636 +334;23452347344633423542;severo-vostocnoe otdelenie N8645 pao sberban +k g. magadan;896868986;98375423895239529987;;23236726352762456346 161;sdgfd sgdsgdsg sdfgds;154-978-292 22;93;17.04.2020;15 000,00;sever +o-vostocnoe otdelenie 8645 pao sberbank;4243972345;347636334;23452347 +344633423542;severo-vostocnoe otdelenie N8645 pao sberbank g. magadan +;896868986;98375423895239529987;;23236726352762456346 162;lkdasj alsdfka sflas;112-031-647 83;61;17.04.2020;15 000,00;severo +-vostocnoe otdelenie 8645 pao sberbank;7707083893;347636334;234523473 +44633423542;severo-vostocnoe otdelenie N8645 pao sberbank g. magadan; +896868986;89686898634243972345;;23236726352762456346
Code used:
#!/usr/bin/env perl my $codepage = 'utf8'; use utf8; binmode(STDOUT, ":$codepage"); binmode(STDOIN, ":$codepage"); use Text::CSV; #get file names: my $fileTable = shift; my @tblRows = (); my @fldNames = ('id', 'fio', 'snils', 'no0', 'data', 'vyplata', 'adres +_bank0', 'no1', 'no2', 'no3', 'adres_bank1', 'no4', 'no5', 'blank', ' +no6'); my %fldNames = (); map { $fldNames{$fldNames[$_]} = $_ } 0..$#fldNames; my $qtyChild = 0; #CSV handle $CSV_H = Text::CSV->new({ sep_char=>";", binary=>1, blank_is_undef=>1, empty_is_undef=>1, allow_whitespace=>0, allow_loose_quotes=>1 ,auto_diag=>1 }); $CSV_H->callbacks(error=>\&csverror2023); if(open my $TBL_H, "<:encoding($codepage)", "$fileTable"){ until($CSV_H->eof){ my $row = $CSV_H->getline($TBL_H); unless($row->[$fldNames{'id'}] !~ /^[[:digit:]]+$/ || $row->[$ +fldNames{'fio'}] =~ /^[[:digit:]]+$/){ map { printf "%12s: %s\n", $fldNames[$_], $row->[$_] } 0.. +$#{$row}; push @tblRows, $row; } print '=' x 100, "\n"; } close $TBL_H; } sub csverror2023{ print join('/', @_), "\n"; my $charpos = $_[2]; my $srcstr = $CSV_H->error_input(); printf "ERROR_INPUT: %s\n", $srcstr; unless($CSV_H->eof){ my $badchar = quotemeta(substr($srcstr, $charpos - 1, 1)); printf "BadChar: (%s)\n", $badchar; my $prestr = substr($srcstr, 0, $charpos - 1); printf "PreMatch: (%s)\n", $prestr; $prestr =~ /$badchar[^$badchar]*$/; printf "MATCH_iNDEXES: (%s-%s)\n", $-[0], $+[0]; printf "BadChar0: (%s)\n", substr($prestr, $-[0], 1); substr($srcstr, $-[0], 0, '\\'); printf "ModifiedString: (%s)\n", $srcstr; } $CSV_H->SetDiag(0); }
Command to execute:
csv-prog.pl csv-file.csv

I added a callback to an error handling function, which gets an invalid record string and fixes the problemmatic areas.
After fixing, I'd like for the fixed string to replace the source string, so it could be reparced in the loop again, but I didn't find any way to pass it back to the loop or to the $CSV_H, so, it replaces the original erroneous string.


In reply to pass repaired csv string from error handler callback function back to the csv's getline loop for reparsing by igoryonya

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.