carlo1973 has asked for the wisdom of the Perl Monks concerning the following question:

Good day :)

I'm sorry to bother everyone again regarding this problem I am having. With the help of everyone here, and variouse resources on the net I think I'm on the cusp of solving problem.

Recap: needing to make a script that would compare the contents of a modified host file in csv format (netbios name, ipaddress) to that of a database that contains similar information. If there are entries missing from the database, the program will report those entries with the information provided from the csv file.

With a lot of help on here, I managed to use script that was given to me and modify it for my needs. The script compiles however but does not appear to run. It just seems to go back to the shell prompt after executing with no errors being displayed.

Here is the current code

#!/usr/bin/perl use strict; use warnings; use Text::CSV_XS; use DBI; my $csv_inputfile = "use_me.txt"; # change me my %data; # hash for storing csv data my $csv = Text::CSV_XS->new; open my $fh, '<', $csv_inputfile or die "can't open $csv_inputfile: $! + \n"; while (my $line = <$fh>) { $line =~ s/^\s+//; # strip leading whitespace next unless $line; # ignore blank lines if ( $csv->parse($line) ) { my ($nbname,$ip) = $csv->fields(); next unless $nbname; $data{$nbname} = { IP => $ip || '', NBNAME => $nbname || '', }; } else { my $error = $csv->error_diag(); print "Parse error: line $. : $error.\n"; } } close $fh; my $user = undef; my $pass = undef; my $dbh = DBI->connect("DBI:mysql:pstsize;host=localhost",$user,$pass, + {RaiseError => 1, AutoCommit => 1},); unless ($dbh) { die "Could not get database handle."; } my $sth = $dbh->prepare( select_data() ); eval { $sth->execute(); }; if ($@) { die "Database error: ", $dbh->errstr; } while ( my ($nbname, $ip) = $sth->fetchrow ) { unless ($ip) { if ($data{$nbname}{'IP'}) { print ".. but CSV file has value: $data{$nbname}{'IP'}{'NBNAME'} +."; } print "\n"; } } $sth->finish; $dbh->disconnect; sub select_data { return <<" END_SQL"; SELECT DISTINCT nbname, ipaddr FROM sizehist END_SQL }

Thank you again :)

This will be my last request for help solving this issue. My internship ends tommorow. :)

I hope to learn the Perl language from the start of my books rather than the hunt and peck method I've been doing thus far.

Carlo Special thanks to scorpio17 for his invaluable assistance!

  • Comment on Comparing csv text file to mysql db in order to look for missing entries within the db.
  • Download Code

Replies are listed 'Best First'.
Re: Comparing csv text file to mysql db in order to look for missing entries within the db.
by scorpio17 (Canon) on May 28, 2009 at 20:12 UTC

    Hey there... I'm sorry you're still having trouble. I guess maybe I didn't understand exactly what it is you're trying to do?

    Maybe this will help:

    Add this at the top, under 'use DBI;':

    use Data::Dumper;

    Then before the 'DBI->connect' line add this:

    print Dumper(\%data), "\n";

    This will dump out all the data read in from your CSV file. You can look at this to make sure it "did something". If your CSV file is huge, copy the first 10 lines or so into another file, and use the small one for testing. Switch to the real one once all the debugging is done.

    Now, I'm not clear whether you want to know if every IP in the CSV file exists in the DB, or if every IP in the DB exists in the CSV file. In my original reply, I assumed the IP in the DB was a primary key, and that you were wanting to check for the existence of other data fields indexed by that IP value. If you're trying to find IPs in the CSV that are missing from the database, that's a different problem! Instead of looping over the DB and comparing to the CSV, you should loop over the CSV and compare against the DB.

    You might try changing your while loop as follows:

    while ( my ($nbname, $ip) = $sth->fetchrow ) { if ($ip) { print "DB has IP=$ip for nbname=$nbname.\n"; } else { print "DB has no IP value for nbname=$nbname.\n"; if ($data{$nbname}{'IP'}) { print "... but CSV file has value: $data{$nbname}{'IP'}.\n"; } } }

    The key here is that this way the loop will print something either way, every time through. That "unless($ip)" in your original code means that if there IS an $ip value, nothing will print. So maybe that's what's happening (which means no errors, right? Yay! You rock!).

      Thanks again everyone and esp scorpio17 for their patience! :) Well I was asked to consult the senior IT guy to get this project done. Unfortunatly he just re-wrote my code. He tried to explain but I wasn't really able to follow as I couldn't see his screen LOL (Unfortunatly my contortionist skills aren't quite to par so bending around his "fortress of death" was a bit difficult. I honestly don't know how he gets in and out of his office with all the towers of computers around him and on his desk LOL)

      Here is the new code:

      #!/usr/bin/perl -w use strict; use DBI; use Net::Ping; use CGI qw(:all) # This script (should) compare a copy of the master host file to the s +izehist database # in order to determine what systems are currently not reporting their + pstsize. my $dbh = DBI->connect( 'DBI:mysql:pstsize' ); my $sth = $dbh->prepare( 'select * from sizehist where ipaddr = ?' ); my $csv_inputfile = "/temp/use_me.txt"; my $NFcount = 0; my $NPcount = 0; my $ping = new Net::Ping; my $time_input_file = "/temp/lasttime.txt"; open my $fh, $time_input_file or die "can't open $time_input_file: $! +\n"; while (my $line = <> ) { chomp ( $line ); my ( $time ) = $line; close $fh print header("text/plain"), start_html("Computers not reporting pstsize"); print <<EndHTML; <h2><b><u><center>Computers not reporting Outlook PST size</h2></b></u +></center> <center>This will also show which of these are not responding to pings + (useful for troubleshooting)<br><br></center> open my $fh, $csv_inputfile or die "can't open $csv_inputfile: $! \n"; while ( my $line = <$fh> ) { chomp( $line ); my ( $name, $ip ) = split /,/, $line; next if $ip =~ /[^\d\.]/; next if $name =~ /^dhcp-/; next if $ip =~ /^192\.168\.8\./; $sth->execute( $ip ); my $row = $sth->fetchrow_hashref; $sth->finish; unless ( $row ) { my $canPing = $ping->ping( $ip, 2 ); ++$NPcount unless $canPing; printf "%s (%s)%s\n", $name, $ip, $canPing ? '' : ' NO PING'; ++$NFcount; } } print "Hosts not found in DB: $NFcount\n"; print "Hosts not responding to PING: $NPcount\n"; print "The host file was last updated on $time\n"; <br> close $fh; EndHTML print end_html;
      Edit

      Well my internship is now complete. The code works! YAY!

      Now I can read those books I baught from begining to end rather than hunt and pecking through the indexes hoping that I will find something useful. I have to admit that even though that this has been trying, it has renewed my interest in studying Perl, MySQL,and PHP :) Thanks again and take care!

Re: Comparing csv text file to mysql db in order to look for missing entries within the db.
by toolic (Bishop) on May 28, 2009 at 19:01 UTC
    does not appear to run
    Be more specific. How does its behavior deviate from your expectations? For example, do you expect it to display some information on STDOUT? Try adding some print statements to narrow down where it is failing.

    I will offer the same advice as before: perhaps the Basic debugging checklist can help.

Re: Comparing csv text file to mysql db in order to look for missing entries within the db.
by bichonfrise74 (Vicar) on May 28, 2009 at 19:27 UTC
    Instead of extracting the data and comparing the results, I would load the data into a temporary database using mysqldump, then create a SQL to compare the results. I would think this would be faster.