Perlbeginner1 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks!


i can run the programme /(see further below) with ease - it gives back the data in a large xml-file. Now i want to store it in a MySQL-database. This is what i have now:

Here there are the data: Data:
0 location:
1 name: mySchool
2 type: HighSchool
3 adress: Paris Champs Ellysees 1
4 description: Ecole Superieur


the database is structured with the following values:
`LOCATION` char(1) NOT NULL,
`NAME` varchar(130) NOT NULL,
`TYPE` varchar(40) DEFAULT NULL,
`ADRESS` smallint(6) NOT NULL,
`DESCRIPTION` varchar(30) NOT NULL,


And here the trials of doing it:

Connecting to a database: use DBI; $dbh = DBI->connect('DBI:mysql:databasename', 'username', 'password' ) || die "Could not connect to database: $DBI::errstr"; # (insert query examples here...) i do the Simple query $dbh->do('CREATE TABLE school_tbl (id INT, val VARCHAR(100))'); with the following values: `LOCATION` char(1) NOT NULL, `NAME` varchar(130) NOT NULL, `TYPE` varchar(40) DEFAULT NULL, `ADRESS` smallint(6) NOT NULL, `DESCRIPTION` varchar(30) NOT NULL, $dbh->do('INSERT INTO school_tbl VALUES(LOCATION, ?)', undef, 'Hello') +; $dbh->do('INSERT INTO school_tbl VALUES(NAME, ?)', undef, 'World'); $dbh->do('INSERT INTO school_tbl VALUES(TYPE, ?)', undef, 'Hello'); $dbh->do('INSERT INTO school_tbl VALUES(ADRESS, ?)', undef, 'World'); $dbh->do('INSERT INTO school_tbl VALUES(DESCRIPTION, ?)', undef, 'Hell +o'); print "Insert the $c rows\n"; $dbh->disconnect();
See the programme that runs with ease - but stores the data in a large file:
#!/usr/bin/perl use strict; use warnings; use diagnostics; use File::Find::Rule; use HTML::TokeParser; # parse HTML-files # Array where all data should be stored in my @schools; my $search_dir='.'; my $out_file='./output.xml'; my @files= File::Find::Rule->file() # ->name('einzelergebnis*.html') # ->in($search_dir); # for my $file (@files) { print "running the job - parsing now: $file!\n"; my %school; # start a new parser-instcance my $p = HTML::TokeParser->new($file) or die "Can't open $file: ($!)" +; while (my $tag = $p->get_tag('div', '/html')) { # first move to the right div that contains the information last if $tag->[0] eq '/html'; next unless exists $tag->[1]{'id'} and $tag->[1]{'id'} eq 'inhalt_ +large'; $p->get_tag('h1'); $school{'location'} = $p->get_text('/h1'); while (my $tag = $p->get_tag('div')) { last if exists $tag->[1]{'id'} and $tag->[1]{'id'} eq 'fusszeile +'; # get the school name from the heading next unless exists $tag->[1]{'class'} and $tag->[1]{'class'} eq +'fm_linkeSpalte'; $p->get_tag('h2'); $school{'name'} = $p->get_text('/h2'); # verify format for school type $tag = $p->get_tag('span'); unless (exists $tag->[1]{'class'} and $tag->[1]{'class'} eq 'sch +ulart_text') { warn "unexpected format: parsing stopped"; last; } $school{'type'} = $p->get_text('/span'); # verify format for address $tag = $p->get_tag('p'); unless (exists $tag->[1]{'class'} and $tag->[1]{'class'} eq 'ein +zel_text') { warn "unexpected format: parsing stopped"; last; } $school{'address'} = clean_address($p->get_text('/p')); # find the description $tag = $p->get_tag('p'); $school{'description'} = $p->get_text('/p'); } } # stpre a refenz on the hash with the data of the actual school in +the Array with all schools push(@schools,\%school); } # output in a file formated with "XML" open(my $fh, '>', $out_file) or die("Error open $out_file ($!)\n"); print $fh "<schools>\n"; for my $school (@schools)#!/usr/bin/perl { print $fh " <school>\n"; print $fh " <name>$school->{name}</name>\n"; print $fh " <location>$school->{location}</location>\n"; print $fh " <type>$school->{type}<type>\n"; print $fh " <address>\n"; for my $address (@{$school->{address}}) { print $fh " <entry>$address</entry>\n"; } print $fh " </address>\n"; print $fh " <description>$school->{description}</description>\n"; print $fh " </school>\n"; } print $fh "</schools>\n"; close($fh); # sanitize # give back as Array each line sub clean_address { my $text = shift; my @lines = split "\n", $text; for (@lines) { s/^s+//; s/s+$//; } return \@lines; }
as mentioned above - the parser-programme works well - but i want to store the data in a mysql-database... this is intended.... How do do that!?

look forward to some ideas!

regards perlbeginner1

Replies are listed 'Best First'.
Re: DBI insert: Need to store results in a MySQL-database (not in a file)
by kcott (Archbishop) on Oct 16, 2010 at 05:58 UTC

    I see you've collected your data into %school. Using that as your data source, you'll need something like the following.

    Do these once:

    use DBI; my $dbh = DBI->connect(...); my $sql_insert = q{ insert into school_tbl (location, name, type, address, description) values (?, ?, ?, ?, ?) }; my $sth = $dbh->prepare($sql_insert);

    For each iteration through your data:

    $sth->execute(@school{qw{location name type address description}});

    And finally, do these once:

    $sth->finish(); $dbh->disconnect();

    NOTE: This code is completely untested and includes no error checking.

    All of the above methods are documented in DBI.

    -- Ken

Re: DBI insert: Need to store results in a MySQL-database (not in a file)
by CountZero (Bishop) on Oct 16, 2010 at 13:14 UTC
    kcott already gave you some valid pointers for your problem.

    However, if you have to do more work on this database or perhaps you need to write more scripts to extract data or manipulate data, then it is a good investment of your time to look into something like DBIx::Class].

    Yes, it takes some learning and it takes some time to set-up the classes for your database, especially if you want to have the relational part right, but once you have done that, using this database becomes a breeze.

    Believe me I speak from experience: before I used DBIx::Class I had to write every time I needed to write a new script for that database, again and again very similar code (connecting to the database, writing SQL, setting up the loops to select, insert, delete, ...). Now all these chores just become standard methods of your class.

    Adding a next record, could become as simple as:

    my $new_school = $schema->resultset('Schools')->create({ location => 1, name => 'mySchool', type => 'HighSchool', adress => 'Paris Champs Ellysees 1', description => 'Ecole Superieur', });

    Searching for a record is equally simple:

    my $rs = $schema->resultset('Schools')->search({name => 'mySchool'});
    Once found, you could extract the fields like this:
    my $school = $rs->first; print $school->adress;
    Updating the record? Even easier!
    $school->adress('New address of the school'); $school->update;

    BTW: for your problem, try adding some or die $dbh->errstr; code to your do statements. You will then be able to see what went wrong.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

      try adding some or die $dbh->errstr
      Or if you're too lazy for that you could also connect with { RaiseError => 1 } which makes your script die whenever some dbi-method fails.
      Hello Kcott, hi CountZero hello Morgon! good evening!

      many many thanks for all! That is great! i love to try out your advices at the weekend.

      many many thanks for all.- I come back and report all my findings.

      regards
      perlbeginner1