fadingjava has asked for the wisdom of the Perl Monks concerning the following question:

monks of wisdom, i am writing this script which takes unique elements out of a text and stores them in an array. These values are then compared to an existing arrayref(a word repository:single_words) whose values come from the database. The point is to enter the values of unique elements of the array found in arrayref in a frequency table in the database. for the values not found in the arrayref the enteries have to be made both in the repository table (single_words) and the frequency table (single_frequency). everything works perfecly in my code till the point i am comparing the values and entering them in single_words. But it starts to screw up when i come to the values not found in the arrayref. Also, the else loop doesnt work at all the way it is written here, if i include it with the comparision if then it works but there are a lot repitions of values. I know that i am doing some mistake with my loops, can anybody correct me? also is there a better way to do it ?? 
my $dbh = DBI->connect("dbi:mysql:concordance","sid" , $password);
my @row = $dbh->selectcol_arrayref("select doc from doc");


@single_uniq = grep { ! $seen{$_} ++ } @single;


my @row = $dbh->selectall_arrayref("select word,word_id from single_wo
+rds");
my $row1 = $dbh->selectall_arrayref("select max(word_id) from single_w
+ords");


$count = ($row1->[0][0]);

$x = ($row1->[0][0]);
$nfound = 0;

single:for ($j=0;$j<=$#single_uniq;$j++){
    $frequency = ($seen{$single_uniq[$j]});
              word:for ($i= 0; $i<= $count ; $i++){
                
              if ("$single_uniq[$j]" eq "$row[0][$i][0]"){
              $found = 1;
                   if ($found){    
                  $dbh->do ("insert into single_frequency values('$row
+[0][$i][0]',$frequency,'$row[0][$i][1]')");
                   }
            
else 
{
   $nfound = ($nfound + 1);
         if ($nfound = 0){
             $dbh->do ("insert into single_words values('','$single_un
+iq[$j]',1)");
            my $max1 = $dbh->selectall_arrayref("select max(word_id) f
+rom single_words");
            $dbh->do ("insert into single_frequency values('$single_un
+iq[$j]',$frequency,'$max1->[0][0]')"); 
            $x = ($x + 1);
}
}
}
}

[download]

Replies are listed 'Best First'.
Re: need help in searching arrayrefs
by halley (Prior) on Feb 12, 2004 at 15:38 UTC

    If you use the word "set" or the word "unique" in the same sentence as the word "array," your design probably needs to be re-thought.

    Arrays are slow to search. Hashes are fast to search. Arrays don't help you keep sets of unique things. Hashes help you keep sets of unique things. Arrays are referenced by integer indexes. Hashes are indexed by any kind of string you find convenient to construct.

    In fact, you're already using a hash, but you might not realize it. That bit where you say $seen{$_} is using a hash which you never declared. (That could cause problems, not declaring or clearing it beforehand, by the way.)

    You look like you're building a concordance or a lexicon. It's all well and good to use this opportunity to learn Perl, and I applaud your desire to solve a problem. You might, however, do a search on google (or perlmonks.org) for the words 'concordance' or 'lexicon' and 'perl'. I expect you'll find quite a few scripts already out there which do the same thing. 

    use strict;                         # complain about errors
my %lexicon;                        # collect unique strings
while (<>) {                        # read each line input
   my @words =                      # gather list of words
      map { s/^\W+|\W+$//g; $_ }    # strip punctuation
      split /\s+/;                  # break line at spaces
   foreach my $word (@words) {      # each word in the line
      $lexicon{$word}++;            # should be counted
   }
}
print "$_: $lexicon{$_}\n"          # show word and count
   foreach (sort keys %lexicon);    # for all words in order

    [download]

    --
    [ e d @ h a l l e y . c c ]

[reply]
[d/l]
[select]
Re: need help in searching arrayrefs
by cchampion (Curate) on Feb 13, 2004 at 01:48 UTC

    There are three things that I noticed and maybe they could make a difference:

    • selectcol_arrayref and selectall_arrayref return an array reference. You are assigning their return value to an array
    • After you assign to @row the value of a fetch, you assign something else, without using that result. Moreover, your second "my @row" declaration will hide the first one. If you use warnings, it will tell you straight away.
    • You are using "select max(word_id)" to get a count. This will work as long as you don't delete any record in your table. You should use "COUNT(*)" instead.

    I don't know if these items are responsible for what you are complaining about, but surely they look suspicious.

    HTH

[reply]
Re: need help in searching arrayrefs
by ChrisR (Hermit) on Feb 12, 2004 at 15:07 UTC
    I'm curious. What version of MySQL and DBI are you using? I haven't seen the methods you are using before: selectall_arrayref I use fetchall_arrayref. And, an array ref should be assigned to a $variable not an @variable. It is a reference to an array not a real array.
[reply]
[d/l]
[select]
      i am using mysql 4.0.17 max-debug and DBI 1.38. i am just using USE:DBI. I realised the thing you said about arrayrefs and changed it , but the problem still remains with the loops.
[reply]

Back to Seekers of Perl Wisdom