athomason has asked for the wisdom of the Perl Monks concerning the following question:

<background>
One of my sites is heavily reliant upon a custom MySQL interface. I've had no problems with my provider to this point, but I like to have a local backup of everything just in case the unthinkable occurs. I have a backup of the filesystem, but I can't get a copy of the database. My first thought was using SELECT INTO, but it's been disabled and the admins aren't inclined to allow it (part of the motivation for my paranoia).
</background>

The solution I've been pursuing is creating a pair of DBI scripts with the dumper useing SHOW TABLES, DESCRIBE, and SELECT to retrieve the data and the restorer using CREATE DATABASE, CREATE TABLE and INSERT to put it back. I looked at DBIx::Copy, but that assumes the table structure exists already; I'd like the scripts to be able to handle an entire arbitrarily shaped database without command-line handholding. The structure returned by DESCRIBE looks somewhat painful and perhaps even insufficient for correctly recreating the table. I've written some code to manage the dump (posted in a reply under this thread), but I'm scared of writing the restorer :-). So, fellow Monks, I submit to you there might be an easier way to do this, hopefully using non-MySQL-specific commands, and if so, I'd love to hear it before doing unnecessary coding.

Replies are listed 'Best First'.
RE: Dumping a Database via DBI
by athomason (Curate) on Jun 29, 2000 at 01:26 UTC
    This is the dumper code I've written; it assumes there's a .my.cnf file in /home/user with connect info. Data::Dumper manages the serialization.
    #!/usr/local/bin/perl -w use strict; use DBI; use Data::Dumper; # Connect to database my $dbh; eval { $dbh = DBI->connect("dbi:mysql:;mysql_read_default_file=/home/user +/.my.cnf", "", "", { RaiseError => 1, PrintError => 0 } ); 1; } or die "Couldn't connect to database: $DBI::errstr"; # Get a list of the tables my @table_names; eval { @table_names = @{$dbh->selectcol_arrayref("SHOW TABLES")}; 1; } or die "Couldn't retrieve list of MySQL tables: $DBI::errstr"; # Get data from each table my (@tables, $table); eval { foreach $table (@table_names) { # Record how the table is layed out my $layout = $dbh->selectall_arrayref("DESCRIBE $table"); # Store the data in a 2D array reference my @fields = map ${$_}[0], @{$layout}; my $data = $dbh->selectall_arrayref("SELECT " . (join ',', @fields) . " FROM $table"); push @tables, [$layout, $data]; } 1; } or die "Failed while obtaining data from $table: $DBI::errstr"; # Print data to STDOUT my %database; @database{@table_names} = @tables; $Data::Dumper::Useqq = 1; $Data::Dumper::Indent = 0; print Data::Dumper->Dump([\%database], ['database']); eval { $dbh->disconnect; 1; } or die "Couldn't disconnect properly from database: $DBI::errstr";
Re: Dumping a Database via DBI
by plaid (Chaplain) on Jun 29, 2000 at 01:33 UTC
    I know you said you don't want any command-line solutions, but have you looked at mysqldump? It comes along with mysql, and lets you do things like 'mysqldump <database> <table> > outputfile'. The output of mysqldump is a set of sql statements which will recreate all tables in the database, and all things contained in the tables.. for a more in-depth description, check out mysqldump --help. I believe that most databases come with a dumper utility like this, and the dumps work well between different SQL databases. You can then recreate the contents of the database by running 'mysql <database> < outputfile'. I hope this fits what you want.
Re: Dumping a Database via DBI
by infoninja (Friar) on Jun 29, 2000 at 01:31 UTC
    The following wouldn't be through DBI (or Perl for that matter), but what about backing the database up using mysqldump ? While it is through the commandline, it's dumps out the contents of the db to SQL statements quite nicely.
    Updated following athomason's comment below:
    Very strange. One might even say icky. Then I guess the best bet is to dump it out the way you're trying.
      Removed by the admins... strange, isn't it? I asked them about how I could get a copy of my database, and they said I couldn't, flat out. Got me to thinking that if I ever tried to switch providers perhaps they wouldn't give me back my data. In any case, this seemed an interesting thing to try through DBI.
        Yikes!

        Reading this made my hair stand on end. I used to work with a client who did exactly that (refused to give his client their data). We left his sorry $#& in a hurry.

        Russ

(jcwren) RE: Dumping a Database via DBI
by jcwren (Prior) on Jun 29, 2000 at 03:30 UTC
    How about getting the sources for the version of mySQL that's running on the system, compiling the mysqldump module, and running it. As near as I can tell from 20 seconds of playing, it doesn't need special priviledges. Simply having access to the database (just like you would through DBI) is sufficient to dump it.

    I realize (also) that it isn't really a Perl solution, but mysqldump is a very clean and trusted way for getting the data out. Not to mention, if you do it right, you can punch the data back into any SQL compatible database. Makes migration to postGres, mSQL, whatever much easier.

    --Chris
Re: Dumping a Database via DBI
by Anonymous Monk on Jun 29, 2000 at 03:43 UTC
    The dumper in phpMyAdmin (http://www.phpwizard.net) works fairly well. I'd suggest that you steal the routines from that program and use them. It should be trivial to port them over to a perl script.
Re: Dumping a Database via DBI
by lhoward (Vicar) on Jun 29, 2000 at 01:40 UTC
    I hate to give you a non-perl option, but....

    Since you're using MySQL qhy not just use its built-in functions for dumping and loading data? In my experience built-in DB tools such as there are often significantly faster than iterating through a DB by hand for replication (typically 10x faster if not more). mysqldump does a great job of dumping a snapshot of a MySQL DB. It can be configured to dump the schemas along with the contents. mysqlimport can then be used to load the dump file back into another MySQL on another server (after you move the dump file over). At the very least, since your source DB is MySQL you can use mysqldump and load from that dump into the other DB. I've used mysqldump and mysqlimport in conjunction with dumps to/from other DB's (most DB's have similar tools) and had very good success. Both tools are very configurable to handle a variety of common methods of formatting the data. IMHO you should alway's use a DB's native import and export functions, if necessary use a perl program to translate the format if both don't use the same format.

    If you are bound-and-determined to do this in perl, you are going to have trouble because there is no standard cross-db way to get a listing of all the tables and to dump their schemas.