MonkPaul has asked for the wisdom of the Perl Monks concerning the following question:
I have written a short script to do a little text mining. I have one file that contains a list of words/terms (new line separated), and a second file that contains a fairly substantial amount of text. I then want to take each term from the first file in turn and count the number of occurances of that term in the second file.
The output would be something like:
Term Number ---------------------- term1 10 term2 1 my term3 16
I am not bothered about the output as yet. The problem I do have, however, is that I am using a regular expression that does not seem to work for a term that consists of multiple words, i.e.
term1 term2 my term 3
The code I have at the moment is:
#!/usr/bin/perl -w use warnings; use strict; use Getopt::Long; my $terms = ""; my $pathway_abstracts = ""; my %word_counts; my $count = 0; my %term_score; my $term_frequency_score = 0; my $word_number = 0; ##################### GET USER INPUT ################################# +########## GetOptions( "pathway_abstracts=s" => \$pathway_abstracts, "terms_file=s" => \$terms ); ################### STORE IN ARRAYS ################################## +########## # store pathway abstract in arrays open(IN, "$pathway_abstracts" ) || die "$!"; my @array_1 = <IN>; close(IN); # store terms in array open(IN2, "$terms" ) || die "$!"; my @array_2 = <IN2>; close(IN2); #################### CREATE HASHES OF TERMS ########################## +########## foreach my $key (@array_2) # assign a score of 0 to each term { chomp($key); $term_score{$key} = 0; } ###################################################################### +########## print("Term\t| "); print("Number\t| "); print("Frequency\n"); print("----------------------------------------------------\n"); for (my $j = 0; $j < @array_2; $j++) # loop through each search + term { chomp($array_2[$j]); my $phenotype_term = $array_2[$j]; # set the search term for(my $i = 0; $i < @array_1; $i++) # loop through each line +in the document { my @word_array = split(/\s/, $array_1[$i]); # split abstra +cts on each word $word_number = $word_number + scalar(@word_array); # find out + how many words are in abstracts foreach my $word (@word_array) # look through each word i +n current line { if($word =~ /\b\Q$phenotype_term\E\b/) # does line contain fi +lter term { $term_score{$array_2[$j]} = $term_score{$array_2[$j]} + 1; +# increment term count } } } $term_frequency_score = $term_score{$array_2[$j]} / $word_number; +# calculate term frequency print($array_2[$j]."\t "); # print the term print($term_score{$array_2[$j]}."\t "); # print the number o +f term occurances print($term_frequency_score."\n"); # print the frequenc +y }
Could anyone please let me know why the \b or \B do not appear to work here in the regualr expression:
$word =~ /\b\Q$phenotype_term\E\b/
Any help is very much appreciated.
many thanks,
MonkPaul.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re: Regex word boundries
by ikegami (Patriarch) on Oct 18, 2007 at 21:11 UTC | |
by MonkPaul (Friar) on Oct 19, 2007 at 12:40 UTC | |
by ikegami (Patriarch) on Oct 19, 2007 at 13:25 UTC | |
by MonkPaul (Friar) on Oct 29, 2007 at 15:24 UTC | |
by ikegami (Patriarch) on Oct 29, 2007 at 15:55 UTC | |
|
Re: Regex word boundries
by duff (Parson) on Oct 18, 2007 at 21:08 UTC | |
|
Re: Regex word boundries
by narainhere (Monk) on Oct 19, 2007 at 08:39 UTC |