Here's a script that I wrote to perform a similar activity. It takes data from a Webmin log, loads the existing session ID's from the DB into a hash, checks each log entry against the hash keys (checking for defined), then injects them to the DB as necessary. I don't currently have any timestamp mechanism in the database to check for "last entry", so it does require that each entry get checked every time. If the logfile was being turned over religiously (I'm not the admin), this wouldn't be an issue. Nevertheless, with over 12k entries in the logfile, it gets done in under 7 seconds.

The script makes use of placeholders (the ?'s), which should solve your problem with the quotes. A lot of the data munging is specific to my needs (particularly that found within massage(), but you should get the gist of everything. The %months hash is a quick hack to translate between months by name and number (there's probably a module to do that, but would it really be less code?). If you can get your hands on a copy, read O'Reilly's Programming the Perl DBI. Hope this helps!

-fuzzyping

#!/usr/bin/perl

use strict;
use DBI;

my $database = "webminstats";
my $server = "localhost";
my $username = "nobody";
my $password = "n0b0d33";
my (@date_parse,$session,$user,$date,$action,$type,$zone,@time_parse,$
+time,@data,%session_hash);
my $masterlog = "/var/www/html/webmin/webmin_master.log";
my $slavelog = "/var/www/html/webmin/webmin_slave.log";
my %months = (
        Jan => "01",
        Feb => "02",
        Mar => "03",
        Apr => "04",
        May => "05",
        Jun => "06",
        Jul => "07",
        Aug => "08",
        Sep => "09",
        Oct => "10",
        Nov => "11",
        Dec => "12"
);

my $dbh = DBI->connect("DBI:mysql:$database:$server","$username","$pas
+sword");
extract_hash();

open(MASTERLOG,"$masterlog");
while (<MASTERLOG>) {
        my @master_fields = split(/ /,$_);
        unless (($master_fields[3] =~ /root/) ||
                ($master_fields[8] =~ /(exec|soa|opts|zonedef|text)/) 
+||
                ($master_fields[9] =~ /(user|forward)/)) {
                massage();
        }
}
close(MASTERLOG);

open(SLAVELOG,"$slavelog");
while (<SLAVELOG>) {
        my @slave_fields = split(/ /,$_);
        unless (($slave_fields[3] =~ /root/) ||
                ($slave_fields[8] =~ /(exec|opts)/)) {
                massage();
        }
}
close(SLAVELOG);

$dbh->disconnect();

sub massage {
        undef $type;
        my @fields = split(/ /,$_);
        my @quick_cheat = ($fields[8],$fields[9]);
        my $action_type = join(" ",@quick_cheat);
        if ($action_type =~ /record/) {
                if ($action_type =~ /create/) {
                        $type = $fields[15];
                } elsif ($action_type =~ /(modify|delete)/) {
                        unless ($fields[19] =~ /ttl/) {
                                $type = $fields[19];
                        } else {
                                $type = $fields[20];
                        }
                }
        }
        $session = $fields[0];
        $user = $fields[3];
        @date_parse = split(/\//,$fields[1]);
        $date_parse[1] = $months{$date_parse[1]};
        $date_parse[0] =~ s/^\[//;
        $date = "$date_parse[2]-$date_parse[1]-$date_parse[0]";
        @time_parse = split(/:/,$fields[2]);
        $time_parse[2] =~ s/\]//;
        $time = "$time_parse[0]:$time_parse[1]:$time_parse[2]";
        $action = "$fields[8] $fields[9]";
        $type =~ s/type\=\'//;
        $type =~ s/'//;
        $zone = $fields[10];
        unless ($session_hash{$session}) {
                inject();
        }
}

sub extract_hash {
        my $select_query = "SELECT session FROM stats";
        my $sth0 = $dbh->prepare($select_query);
        $sth0->execute();
        while (@data = $sth0->fetchrow_array) {
                $session_hash{$data[0]} = 1;
        }
}

sub inject {
        my $insert_stmt = "INSERT INTO stats (date,
                                                time,
                                                user,
                                                session,
                                                action,
                                                type,
                                                zone) VALUES (?,?,?,?,
+?,?,?)";
        my $sth2 = $dbh->prepare($insert_stmt);
        $sth2->execute($date,$time,$user,$session,$action,$type,$zone)
+;
}

[download]

In reply to Re: Writing special characters to mysql DB by fuzzyping
in thread Writing special characters to mysql DB by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.