parsing error?

bagelesque has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to read a list of URLs from a textfile and have them put into a format for easy inserting into a MySQL dbase. Only problem is after the URL I get a ^M before the "', 'General');" string.

#!/usr/bin/perl

$input = @ARGV[0];
open (INFILE, $input) || die "Unable to open @ARGV[0]!\n";
@lines = <INFILE>;
close(INFILE);

$output = ">output.txt";
open (OUTFILE, $output) || die "Unable to open output.txt!\n";

$frontfill = "INSERT INTO urls (URL, CATEGORY) VALUES ('";
$backfill = "', 'General');";

foreach (@lines)
{ chomp($lines[$x]);
  print OUTFILE $frontfill;
  print OUTFILE $lines[$x];
  print OUTFILE $backfill;
  print OUTFILE "\n";
  $x++;
}
close(OUTFILE);
[download]

Comment on parsing error? Download Code

Replies are listed 'Best First'.
Re: parsing error? by miyagawa (Chaplain) on Dec 03, 2001 at 16:04 UTC
You should do `chomp` twice or do `$lines[$x] =~ tr/\r\n//d` -- Tatsuhiko Miyagawa miyagawa@cpan.org	[reply] [d/l] [select]
Re: Re: parsing error? by MZSanford (Curate) on Dec 03, 2001 at 17:02 UTC
Because `chomp()` looks at `$/` and removes the last character only if it matches, i don't think using it twice will help. I think the `tr///` code is a better idea, or you could use `chomp()` followed by `chop()`. i had a memory leak once, and it ruined my favorite shirt.	[reply] [d/l] [select]
Re: Re: parsing error? by bagelesque (Initiate) on Dec 03, 2001 at 16:44 UTC
works like a champ. can't believe i didn't think of that one. that's what i get for PERL'ing when i'm sleepy grin	[reply]
Re: parsing error? by larryk (Friar) on Dec 03, 2001 at 15:55 UTC
if you `binmode INFILE;` after you open it then you can substitute out the dodgy chars before you insert to the DB. I had the same problem with a corrupt logfile line on a win32 system which had a `^Z` in it. perl just jumped straight out of the while loop without so much as a by-your-leave! Hope this helps larryk perl -le "s,,reverse killer,e,y,rifle,lycra,,print" Will code for food - looking for work - London - CV	[reply] [d/l] [select]
Re: parsing error? by Beatnik (Parson) on Dec 03, 2001 at 18:26 UTC
Hardly anything to do with a parsing error but I suggest using `foreach (@lines) { #chomp; tr/\r\n//d; #like miyagawa suggested print OUTFILE $frontfill,$_,$backfill,"\n"; }` [download] Greetz Beatnik ... Quidquid perl dictum sit, altum viditur.	[reply] [d/l]
Re: parsing error? by MZSanford (Curate) on Dec 03, 2001 at 18:56 UTC
Appears the other posts have the answers, but i would also suggest using DBI, DBD::mysql and DBI Placeholders ...that way the mysql loading can be done at the same time as the parsing. Just a thought. i had a memory leak once, and it ruined my favorite shirt.	[reply]
Re: parsing error? by Fastolfe (Vicar) on Dec 03, 2001 at 21:58 UTC
Sounds like your input file has non-native newlines in it. Other posters suggest ways of compensating for this in your code. I might suggest that you fix this problem at the source by repairing the transfer mechanism you use so that ASCII newline conventions are translated between the different OS's. Generally transferring a file via FTP in "ASCII" mode will do this, as should HTTP User-Agents and even Samba. If you're using a proprietary transfer process to get this file to your system, you should look into augmenting it with support for newline translations.	[reply]