in reply to Replacing a character in a file

I think that we could help you much more if you could please post an actual, non-confidential (sanitized) example of one of the data-file lines that you are speaking of. If you "don't know where the characters are coming from," the first order of business is to find out! It is unfortunately very easy to write a program that, in a sincere effort to "avoid crashing," actually conceals an underlying problem with the data (source). Let's get to the bottom of this. Let us help you.

Replies are listed 'Best First'.
Re^2: Replacing a character in a file
by Broggy69 (Novice) on Jan 03, 2018 at 21:44 UTC

    Lots of replies about the special character. The original data is as follows: (Copied before file was imported in MS-Access) I will also add this isn't for production, just wanting to learn, so in this case I do not care about the character. My guess is the file sent to me came from Excel which is nortorious for adding unwanted characters. Or replacing a "-" with a character.

    My text file before import, in MS-Access, and after export looks like

    101,Thomasville 104,Tuscaloosa 105,Opelika 108,Tucker 110,Jacksonville 111,Lakeland 112,Miami 113,Villa Rica 114,Lafayette 115,New Orleans 118,Houston 120,Jamestown 124,Lynchburg Bakery 126,Tyler 127,Phoenix 128,Tolleson 129,Fontana 130,Lenexa 133,Modesto 134,Henderson 136,Knoxville 140,Bradenton 147,DKB Bakery 149,Baton Rouge 152,El Paso Bakery 161,Oxford 165,Lepage - Park St 166,Lepage - Cedar St 167,Lepage - Brattleboro 170,San Antonio 180,Bailey Street 181,Goldsboro 184,Norfolk 187,Batesville 188,Denton 190,Leeland 191,Newton 192,Savannah 193,Bardstown 321,Alpine 355,Montgomery 423,Lynchburg DC 430,Bridgeton DC 440,Reynolds DC 447,DKB DC 452,El Paso DC 456,Sacramento 457,Hope Mills 488,Memphis

    I will give the regex a try.

    2018-01-05 Athanasius removed paragraph tags from data, and added code tags

      Please use <code> and </code> tags for your data samples. Otherwise it is almost impossible to read.

        I used the tags. I am not sure why they didn't work. I eve previewed the message and everything looked good

      Try creating the file directly with both ADO and ODBC connections.

      #!perl use strict; use DBI; use Text::CSV; use Devel::Peek; my $dir = 'C:/GBowl/strawberry32/Scripts/'; my $db_file = $dir.'Hello.accdb'; my $outfile = $dir.'Plantfile.txt'; my $dsn; $dsn = 'dbi:ADO:Provider=Microsoft.ACE.OLEDB.12.0;Data Source='; #$dsn = 'dbi:ODBC:driver=Microsoft Access Driver (*.mdb, *.accdb);dbq= +'; my $dbh = DBI->connect("$dsn$db_file", { RaiseError => 1 } ) or die $DBI::errstr; my $sql = ' SELECT Company_Code,Name FROM Plants ORDER BY Company_Code'; my $rs = $dbh->selectall_arrayref($sql); open my $out,'>',$outfile or die "Could not open $outfile : $!"; my $csv = Text::CSV->new ( { binary => 1 } ) or die Text::CSV->error_diag(); $csv->eol ("\n"); for (@$rs){ $csv->print($out,$_); my $line = join "\t",@$_; # check non-ascii if ($line =~ /[^\x09\x20-\x7E]/){ Dump $line; } } close $out;
      poj