Trying to make a string generated like google

ultranerds has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I'm trying to make a string show up like:

Some test bla bla .... and now we carry on, with more keywords...

Basically, the string would look something like:

Some test bla bla. We wanna add some more content in here, just as an example - and now we carry on, with more keywords etc

Here is my current code:

    my $keywords = q|fait une petite St-Laurent|;
    my @keywords = split / /, $keywords;

    $text =~ s|<!--start_tag-->\s*.*?<!--end_tag-->||g; #remove all th
+e citations
    $text =~ s|&lt;|<|g;
    $text =~ s|&gt;|>|g;
    $text =~ s|<\s*.*?>||g; #remove any html
    $text =~ s|\[\s*.*?\]||g; #remove any bbcode tags

    my $strings;

    use Data::Dumper;
    
    my @split = split / /, $text;

    my $number     = 0;
    my $hash_count = 0;
    foreach (@split) {
        $back .= "$_ ";
        if (length($back) >= 250) {
#            print qq|Adding to: \$string->{$number} \n|;        
            $strings->{$number}->{value} = $back;
            $strings->{$number}->{count} = get_word_count_match($back)
+;            
            $back = undef;
            $number++;
        }
    }


    print Dumper($strings);


sub get_word_count_match {

    my $text = $_[0];
    
    my $total_words_found;
    foreach (@keywords) {

        my $count =0;
        $count++ while $text =~ /\Q$_/g;

        $total_words_found = $total_words_found + $count;

    }

    return $total_words_found || 0 ;

}
[download]

What this does, is generate a hashref, like so:

user@serverdev admin $ perl test.cgi
$VAR1 = {
          '1' => {
                   'count' => 3,
                   'value' => 'large. La température de l\'eau durant 
+l\'été ne dépasse pas 3 ou 4°c et lorsque le vent se met de la partie
+ vaut mieux être bien habillé...
Cet été, j\'ai fait une petite excursion en kayak de mer dans la régio
+n des Bergeronnes (à l\'Est de Tadoussac). '
                 },
          '0' => {
                   'count' => 2,
                   'value' => '
        Salut Memphre,
Je ne connais pas les dates de fin des excursions pour aller voir les 
+baleines dans le St-Laurent. Mais je ne crois pas qu\'une ballade en 
+bateau au large en octobre soit très agréable. Déjà, durant l\'été, i
+l peut faire très froid au '
                 },
          '2' => {
                   'count' => 0,
                   'value' => 'Nous avons vu plusieurs baleines: rorqu
+als communs, marsouins et bélugas. C\'est vraiment impressionnant de 
+voir les baleines de si près! Entre l\'excursion en bateau ou celle e
+n kayak, je te conseille fortement celle en kayak. L\'expérience est 
+beaucoup '
                 }
        };
[download]

Now, this is exactly what I want.

After this, I need to then grab the "phrases" which has the highest value for "count" .. in this case, it would be 1 and 0.

Can anyone suggest to me how I could accomplish this?

I know I could enter the values into a mySQL table, and then do an ORDER BY to get the 2 highest values - but this is going to be run on about 20 strings per page, so I would imagine that would be a little slow (especially seeing as we are doing about 2 million+ page views a week, and about 200,000 searchs =))

TIA!

Andy

Comment on Trying to make a string generated like google Select or Download Code

Replies are listed 'Best First'.
Re: Trying to make a string generated like google by davorg (Chancellor) on Jul 20, 2009 at 10:39 UTC
`my @sorted_phases = sort { $b->{count} <=> $a->{count} } values %$strings;` [download] You'll then have an array of your hashes ordered by the "count" value. The ones with the highest count will be at the start of the array. -- See the Copyright notice on my home node. Perl training courses	[reply] [d/l]
Re^2: Trying to make a string generated like google by ultranerds (Hermit) on Jul 20, 2009 at 10:43 UTC
OMG - your a star! I've been trying to get something like that to work for ages, and you did it in one line - thanks, it doesn't exactly what I need :) Cheers Andy	[reply]
Re^2: Trying to make a string generated like google by ultranerds (Hermit) on Aug 10, 2009 at 14:01 UTC
Hi, I'm trying to do something a bit more complex now: `my @sorted_phrases = sort { $b->{sort_number} <=> $a->{sort_number +}, $b->{count} <=> $a->{count} } values %$strings;` [download] ..instead of just: `my @sorted_phrases = sort { $b->{count} <=> $a->{count} } values %$strings;` [download] Basically, what I need to do - is : 1) Sort by the value "count" as they were done before 2) If the 2nd array element has a value of "0" for "count", then it will order them by sort_number thereafter. I've tried all kinda things - including this messy bit of code - but I can't seem to get it working :( my @sorted_phrases_2; # my $i = 0; foreach (my $i = 0; $i <= $#sorted_phrases; $i++) { local $_ = $sorted_phrases[$i]; if ($i == 1) { print "BLA BLA XXX - \"$_->{count}\" " . Dumper($_); if ($_->{count} < 1) { my $chosen_id_now = $sorted_phrases[0]->{sort_number} ++ 1; print qq\|<br />Current sort was: $sorted_phrases[0]->{ +sort_number} , and new one is $chosen_id_now <br />\|; foreach my $tmp (@sorted_phrases) { if ($tmp->{sort_number} == $chosen_id_now) { push @sorted_phrases_2, $tmp; } } } } else { push @sorted_phrases_2, $_; } } @sorted_phrases = @sorted_phrases_2; [download] Thanks for any ideas - as I'm stumped :( Cheers Andy	[reply] [d/l] [select]
Re^3: Trying to make a string generated like google by davorg (Chancellor) on Jun 30, 2010 at 07:50 UTC
I've just logged on for the first time for almost a year and saw your unanswered question. It's really quite simple. `my @sorted_phrases = sort { $b->{count} <=> $a->{count} or $b->{sort_number} <=> $a->{sort_number} } values %$strings;` [download] It's really worth reading the Perl FAQ. See How do I sort an array by (anything)? -- See the Copyright notice on my home node. Perl training courses	[reply] [d/l]
Re: Trying to make a string generated like google by moritz (Cardinal) on Jul 20, 2009 at 10:50 UTC
Maybe you want a "real" search engine instead, like KinoSearch or Plucene?	[reply]
Re^2: Trying to make a string generated like google by ultranerds (Hermit) on Jul 20, 2009 at 11:13 UTC
Hi, We are already using Lucine for the searches, as we have a lot of posts (and this has prooved the most effecient for what we need) Thanks for the suggestion though =) Cheers Andy	[reply]