Re: Unable to load entire CSV file into DB

Something for you to investigate:

Are you sure that the file is uploaded completely?
Are you checking for database errors?
Have you read Before asking a database related question ...?
Did you try that SQL statement outside your script? I mean, manually, from the mysql monitor?
Does the table structure correspond to the CVS structure?

If the answer to all the above questions is "yes", then I am prepared to accept that you might have a Perl problem. Otherwise, I should address you to a a more appropriate place where to ask your question.

Moreover, you are asking for a database related error, and then you are showing us an uploading script.

Is that relevant?

If not, why don't you skim the example to the only thing that matters?

Comment on Re: Unable to load entire CSV file into DB

Replies are listed 'Best First'.
Re^2: Unable to load entire CSV file into DB by bradcathey (Prior) on Nov 06, 2004 at 17:52 UTC
Thanks for the tips. `Are you sure that the file is uploaded completely?` [download] Yes, in all cases, it's all there `Are you checking for database errors?` [download] I'm using `{ RaiseError => 1}` and `$dbh->trace(2)` and no errors. `Have you read Before asking a database related question ...?` [download] I have, but just re-read it. `Did you try that SQL statement outside your script? I mean, manually, +from the mysql monitor?` [download] Did not. Working on a remote shared web server. `Does the table structure correspond to the CVS structure?` [download] Dead on! Again, some of the file is importing. It seems like there is some hidden character in the CSV. Per your suggestion, I will ask in a more 'targeted' forum. Thanks! —Brad "Don't ever take a fence down until you know the reason it was put up." G. K. Chesterton	[reply] [d/l] [select]
Re^3: Unable to load entire CSV file into DB by tachyon (Chancellor) on Nov 07, 2004 at 04:13 UTC
I expect the file format will be the issue, quite possibly the line endings. The first simple thing in your upload is to do this: `$buffer =~ s/[\r\n]+/\n/g;` [download] to standardise all your line endings. You could use \r if you want but \n is the default, the standard .... and it's less typing in your LOAD DATA. If that does not work then I suggest you use perl to read and validate the uploaded file before you pass it to mysql. I too suggest trying to load the file manually. You could download the file and try loading it into a local MySQL DB if you don't have access to the remote one. cheers tachyon	[reply] [d/l]
Re^4: Unable to load entire CSV file into DB by bradcathey (Prior) on Nov 07, 2004 at 12:46 UTC
Great suggestion, I've written this in as one more safeguard. However, I don't understand all that I'm seeing when I check for ascii or hex values. Here's a dump of a file that looks like this: `,AL,2 ,AK,4` [download] in ascii `A--65 L--76 ,--44 2--50 --13 ,--44 A--65 K--75 ,--44 4--52 --13` [download] What is the blank spot? Even if it's a new line, shouldn't it have a value? Just curious. —Brad "Don't ever take a fence down until you know the reason it was put up." G. K. Chesterton	[reply] [d/l] [select]
Re^5: Unable to load entire CSV file into DB by steves (Curate) on Nov 07, 2004 at 13:10 UTC
Re^6: Unable to load entire CSV file into DB by bradcathey (Prior) on Nov 08, 2004 at 00:02 UTC
Re^3: Unable to load entire CSV file into DB by Anonymous Monk on Nov 06, 2004 at 23:43 UTC
Did not. Working on a remote shared web server. Don't you think that this very step could tell you in which direction to look? I would try to do the testing in a local machine. Installing MySQL is not rocket science and your table does not seem to have a million records. So, instead of guessing, why don't you test it?	[reply]