artist has asked for the wisdom of the Perl Monks concerning the following question:

I am looking for popular practices for updating the data to database. I am receving data in text files and need to update to mysql database as soon as the files are changed. There are multiple files in various directories and new files (with current date in their name) are created and updated at anytime of the day. Often it happens that only few records are changed in files. I am planning to use Class::DBI and Template for presentation if that matters. Other tools available are 'cron', 'mysql load data' etc...

Besides this, I am also looking for a way to export the data into another text file, as soon as a particular table is changed.

Thanks.

--Artist

Replies are listed 'Best First'.
Re: Update to Database
by g0n (Priest) on Oct 11, 2005 at 17:47 UTC
    This is approximately the sort of application that I intended Data::Sync to be useful for artist, you could use DBD::CSV & DBD::MySQL handles with it. OTOH, if it doesn't do what you need, I'd love to know why so I can address the shortcomings.

    Update: v0.04 includes attribute/record hashing, so a record is only written to the target if it has changed in the source, something your OP suggests you might need. I've got some other functionality to include in it, so I haven't uploaded it to CPAN yet, but if you want to try it out, please /msg me.

    --------------------------------------------------------------

    $perlquestion=~s/Can I/How do I/g;

[reply]
Re: Update to Database
by TedPride (Priest) on Oct 11, 2005 at 17:40 UTC
    Do you have control over file changes? The easiest way to do this is with something event-driven, where a file upload script immediately calls your database updater. Short of that, you can set up a cron job to check each directory for files modified on or after your last check (-m $path >= $timestamp), then process the resulting matches.

    As for exporting your data, you can do the usual select and print, or if you need to back up the entire database, you might want to look at mysqldump:

    http://dev.mysql.com/doc/mysql/en/mysqldump.html

[reply]
Re: Update to Database
by tirwhan (Abbot) on Oct 12, 2005 at 08:24 UTC
    If you're using Linux with kernel 2.6.13 or later you can use Linux::Inotify to monitor file/directory changes.
[reply]

Back to Seekers of Perl Wisdom