I revised my code from an earlier post and look to my fellow monks for constructive criticism on better ways to do this. This is a rewritten version of the query program for parsing NT logs into NCSA format (yes, I know that IIS logs to NCSA, but these are existing logs rather than logs for new sites).
#!/usr/bin/perl #!/usr/bin/perl -w # -*- perl -*- @(#) $Id$ # # $Revision$ # $Date$ # $Author$ # $Locker$ # $RCSfile$ # $State$ # $Source$ require 5.005; use strict; umask 002; my $prog_name = $0; my $log_basedir = '/webstat/NT/'; my $log_dir = ''; my $sites_dat = '/etc/weblog/nt-sites.dat'; my $syscmd_webalizer_wrapper = '/opt/webalizer/bin/webalizer-wrapper'; # The parse2ncsa command takes a directory name as a parameter, # and parses all of the logs in that directory in chronological # order, converting to NCSA format wherever necessary. It sends # the NCSA common/combined log format to STDOUT. my $syscmd_parse2ncsa = '/opt/webalizer/bin/parse2ncsa'; chdir $log_dir or die "Could not change to $log_dir"; my $domain; my $webserver; my $w3svc; my $ip_addr; my $line; open(SITES_DAT,"<$sites_dat") or die; while (defined ($line = <SITES_DAT>)) { $line =~ /^#/ and next; chomp $line; ($domain, $webserver, $w3svc, $ip_addr) = split(/\t/,$line,4); $0 = $prog_name . " - processing $domain from $webserver/$w3svc"; my $log_dir = join('/', $log_basedir, $webserver, $w3svc); open(NCSA_STREAM,"$syscmd_parse2ncsa $log_dir $domain |") or next; open(WEBALIZER, "|$syscmd_webalizer_wrapper $domain") or next; while (defined ($line = <NCSA_STREAM>)) { $line =~ tr/\000//d; print WEBALIZER $line while defined ($line = <NCSA_STREAM>); } close(WEBALIZER); close(NCSA_STREAM); } close(SITES_DAT); my $datakey = "domain"; my $driver = 'mysql'; my $database = 'webstat'; my $table = 'sites'; my $user = 'withheld'; my $password = 'withheld'; my $dbh; my @Fields = qw( priority domain server_name server_type log_code log_ +location ip_addr username timestamp ); my $button = (defined($req->param("button"))) ? $req->param("button") +: "Default"; my %FormData = getformdata(); sub db_connect() { my $dsn = "DBI:$Driver:$DataBase"; $dbh = DBI->connect($dsn, $User, $Password, {RaiseError => 1, Auto +Commit => 1}); } sub db_disconnect() { $dbh->disconnect; } my $query = "select * from $Table where $DataKey = ?"; my $sth = $dbh->prepare($query); $sth->execute($FormData{$DataKey});

In reply to Efficiency and Zen log parsing by tekniko

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.