in reply to Writing special characters to mysql DB

Here's a script that I wrote to perform a similar activity. It takes data from a Webmin log, loads the existing session ID's from the DB into a hash, checks each log entry against the hash keys (checking for defined), then injects them to the DB as necessary. I don't currently have any timestamp mechanism in the database to check for "last entry", so it does require that each entry get checked every time. If the logfile was being turned over religiously (I'm not the admin), this wouldn't be an issue. Nevertheless, with over 12k entries in the logfile, it gets done in under 7 seconds.

The script makes use of placeholders (the ?'s), which should solve your problem with the quotes. A lot of the data munging is specific to my needs (particularly that found within massage(), but you should get the gist of everything. The %months hash is a quick hack to translate between months by name and number (there's probably a module to do that, but would it really be less code?). If you can get your hands on a copy, read O'Reilly's Programming the Perl DBI. Hope this helps!

-fuzzyping

#!/usr/bin/perl use strict; use DBI; my $database = "webminstats"; my $server = "localhost"; my $username = "nobody"; my $password = "n0b0d33"; my (@date_parse,$session,$user,$date,$action,$type,$zone,@time_parse,$ +time,@data,%session_hash); my $masterlog = "/var/www/html/webmin/webmin_master.log"; my $slavelog = "/var/www/html/webmin/webmin_slave.log"; my %months = ( Jan => "01", Feb => "02", Mar => "03", Apr => "04", May => "05", Jun => "06", Jul => "07", Aug => "08", Sep => "09", Oct => "10", Nov => "11", Dec => "12" ); my $dbh = DBI->connect("DBI:mysql:$database:$server","$username","$pas +sword"); extract_hash(); open(MASTERLOG,"$masterlog"); while (<MASTERLOG>) { my @master_fields = split(/ /,$_); unless (($master_fields[3] =~ /root/) || ($master_fields[8] =~ /(exec|soa|opts|zonedef|text)/) +|| ($master_fields[9] =~ /(user|forward)/)) { massage(); } } close(MASTERLOG); open(SLAVELOG,"$slavelog"); while (<SLAVELOG>) { my @slave_fields = split(/ /,$_); unless (($slave_fields[3] =~ /root/) || ($slave_fields[8] =~ /(exec|opts)/)) { massage(); } } close(SLAVELOG); $dbh->disconnect(); sub massage { undef $type; my @fields = split(/ /,$_); my @quick_cheat = ($fields[8],$fields[9]); my $action_type = join(" ",@quick_cheat); if ($action_type =~ /record/) { if ($action_type =~ /create/) { $type = $fields[15]; } elsif ($action_type =~ /(modify|delete)/) { unless ($fields[19] =~ /ttl/) { $type = $fields[19]; } else { $type = $fields[20]; } } } $session = $fields[0]; $user = $fields[3]; @date_parse = split(/\//,$fields[1]); $date_parse[1] = $months{$date_parse[1]}; $date_parse[0] =~ s/^\[//; $date = "$date_parse[2]-$date_parse[1]-$date_parse[0]"; @time_parse = split(/:/,$fields[2]); $time_parse[2] =~ s/\]//; $time = "$time_parse[0]:$time_parse[1]:$time_parse[2]"; $action = "$fields[8] $fields[9]"; $type =~ s/type\=\'//; $type =~ s/'//; $zone = $fields[10]; unless ($session_hash{$session}) { inject(); } } sub extract_hash { my $select_query = "SELECT session FROM stats"; my $sth0 = $dbh->prepare($select_query); $sth0->execute(); while (@data = $sth0->fetchrow_array) { $session_hash{$data[0]} = 1; } } sub inject { my $insert_stmt = "INSERT INTO stats (date, time, user, session, action, type, zone) VALUES (?,?,?,?, +?,?,?)"; my $sth2 = $dbh->prepare($insert_stmt); $sth2->execute($date,$time,$user,$session,$action,$type,$zone) +; }