Processing a file into mySQL and writing a results file

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have a very limited knowledge of Perl, but thought it to be the right tool for this job:
1)Search for a file in a specific directory with name such as filename001.txt
2) Check that the previous file processed succesfully was filename.000.txt, if not, write to an error file.
3) Process the file by inserting its contents into a mySQL database. When done, write to a file an OK (or true indicator) with 001 to relate it to the file processed. If process to mySQL unsuccesful try to reprocess. If still unsuccesful write to an error file.

This would run as a cron on a Unix machine every hour. The file to be processed is a database update being ftped from a Windows server via an app that checks the output files generated above to determine which file to upload (i.e. filename001 instead of filename000), if filename 001 is not yet OK'ed, it will reupload it. It will not upload filename002 until 001 is processed correctly, and so on.

So, I need to know if Perl IS the right tool for this. Also need to know if someone is willing to help me through this (a lot of handholding probably). Thanks in advance and kudos on a superb Perl site.

Comment on Processing a file into mySQL and writing a results file

Replies are listed 'Best First'.
Re: Processing a file into mySQL and writing a results file by n3dst4 (Scribe) on Jan 08, 2002 at 23:04 UTC
Perl is certainly a right tool for the job. It's not a particulary complex scenario, but you'll need to look into a few Perl subjects: 1) directory globs, for finding out what matching files exist 2) open(), to read the log file and the data file 3) DBI, to insert into the database What level of programming experience (and specifically, Perl) do you have? In any case, I think your first step should be to re-write your bullet points above in a more logical, mechanical way. Then the job is just to translate them into Perl (easy, ain't it? :-) If you don't feel you can attempt writing the actual code just yet, post pseudocode and we'll take it from there. Incidentally, if there are only three digits, you're going to loop after 41 days running. Will that be a problem?	[reply]

Replies are listed 'Best First'.

Re: Processing a file into mySQL and writing a results file
by n3dst4 (Scribe) on Jan 08, 2002 at 23:04 UTC

1) directory globs, for finding out what matching files exist
2) open(), to read the log file and the data file
3) DBI, to insert into the database

What level of programming experience (and specifically, Perl) do you have? In any case, I think your first step should be to re-write your bullet points above in a more logical, mechanical way. Then the job is just to translate them into Perl (easy, ain't it? :-) If you don't feel you can attempt writing the actual code just yet, post pseudocode and we'll take it from there.

Incidentally, if there are only three digits, you're going to loop after 41 days running. Will that be a problem?

[reply]