in reply to Re^7: how to use Algorithm::NaiveBayes module
in thread how to use Algorithm::NaiveBayes module

HI~I have modified the code as you said, but it still deal with the text as a whole, and export the tone of the whole text, the output is like this:

%neutral:given this focus execut area can control continu provid high valu product lowest total cost a detail discuss risk uncertainti caus actual result event differ materi forward look statement incluthese forward look statement general identifi word believe project expect anticipate estimate intend certain statement management discuss analysi md&a pure histor information includ estimates projections statement relat busi plans objectives expect oper results assumpt upon statement based forward look statements within mean privat secur litig reform act 1995 section 27a secur act 1933 section 21e secur exchang act 1934.

the out put is the tone of the whole text and all the sentence in which the full stop of each sentence were removed from the text. you may need the code,the code is like this: 
#!/usr/bin/perl
use warnings;
use Algorithm::NaiveBayes;

my $pos_file = '/Users/Agnes/Documents/positive.TXT';
my $neg_file = '/Users/Agnes/Documents/negative.txt';
my $neu_file = '/Users/Agnes/Documents/neutral.txt';

my $categorizer = Algorithm::NaiveBayes->new;
my $fh;

open($fh,"<",$pos_file) or die "Could not open $pos_file: $!";
while (my $sentence = <$fh>) {
    chomp $sentence;
    my @words = split(' ',$sentence);
    my %positive;
    $positive{$_}++ for @words;
    $categorizer->add_instance(
    attributes => \%positive,
    label => 'positive');
}
close($fh);

open($fh,"<",$neg_file) or die "Could not open $neg_file: $!";
while (my $sentence = <$fh>) {
    chomp $sentence;
    my @words = split(' ',$sentence);
    my %negative;
    $negative{$_}++ for @words;
    $categorizer->add_instance(
    attributes => \%negative,
    label => 'negative');
}
close($fh);

open($fh,"<",$neu_file) or die "Could not open $neg_file: $!";
while (my $sentence = <$fh>) {
    chomp $sentence;
    my @words = split(' ',$sentence);
    my %neutral;
    $neutral{$_}++ for @words;
    $categorizer->add_instance(
    attributes => \%neutral,
    label => 'neutral');
}
close($fh);

$categorizer->train;

my $sentence_file = '/Users/Agnes/Documents/process_sentence.txt';

open($fh,"<",$sentence_file) or die "Could not open $sentence_file: $!
+";
while (my $sentence = <$fh>) {
    chomp $sentence;
    my @words = split(' ',$sentence);
    my %test;
    $test{$_}++ for @words;
    my $probability = $categorizer->predict(attributes => \%test);
    if ( $probability->{positive} > 1/3 ) {
        print "%positive:$sentence\n";
    }
    if ( $probability->{negative} > 1/3 ) {
    print "%negative:$sentence\n";
    }
    if ( $probability->{neutral} > 1/3 ) {
        print "%neutral:$sentence\n";
    }
   }
close($fh);
# if ( $probability->{negative} > 1/3 ) {
#print "%negative\n";
#}
#if ( $probability->{neutral} > 1/3 ) {
#    print "%neutral\n";
#}

[download]
I do not know what should I do next. It will be very kind of you to help me to solve this problem. Thank you!!

Replies are listed 'Best First'.
Re^9: how to use Algorithm::NaiveBayes module
by tangent (Parson) on Apr 26, 2014 at 12:51 UTC
    It looks like the sentences in your file are not separated by new lines, or at least not by new lines recognized by the perl script. If you are on a Mac this can happen when you export files from Excel or Filemaker - the new lines are in the Classic Mac format (carriage return) rather than the Unix format (line feed). Try this to see: 
    my $count = 0;
open($fh,"<",$sentence_file) or die "Could not open $sentence_file: $!
+";
while (my $sentence = <$fh>) {
    chomp $sentence;
    $count++;
    print "Sentence $count\n";
}
close($fh);

    [download]
    If it only counts one sentence then this is your problem. If you use a text editor like BBEdit you can change the format of your files by opening Edit->Document Options and choosing Unix. Alternatively, you can set the line break that Perl sees at the top of your script (before opening the files): 
    #!/usr/bin/perl
use warnings;
use Algorithm::NaiveBayes;

local $/ = "\015";

    [download]
[reply]
[d/l]
[select]
      Thank you so much for your help!! I almost got the result I want. One last question, in my program, I have three categories which are positive, negative and neutral. I use this code to decide which categories a sentence belongs to 
      if ( $probability->{positive} > 1/3 ) {
        print "%positive:$sentence\n";
    }
    if ( $probability->{negative} > 1/3 ) {
    print "%negative:$sentence\n";
    }
    if ( $probability->{neutral} > 1/3 ) {
        print "%neutral:$sentence\n";
    }

      [download]
      however, sometimes one sentence may belong to two category, for example, if the probability of positive is 0.34 and the probability of neutral is 0.35, then this sentence will belong to both positive and neutral. I wrote this code to try to solve this problem: 
      if( $probability->{positive} > $probability->{negative} > $probability
+->{neutral})
    {
    print"%positive:$sentence\n";
    }
    if( $probability->{positive} > $probability->{neutral} > $probabil
+ity->{negative})
    {
        print"%positive:$sentence\n";
    }
    if( $probability->{negative} > $probability->{positive} > $probabi
+lity->{neutral})
    {
        print"%negative:$sentence\n";
    }
    if( $probability->{negative} > $probability->{neutral} > $probabil
+ity->{positive})
    {
        print"%negative:$sentence\n";
    }
    if( $probability->{neutral} > $probability->{positive} > $probabil
+ity->{negative})
    {
        print"%positive:$sentence\n";
    }
    if( $probability->{neutral} > $probability->{negative} > $probabil
+ity->{positive})
    {
        print"%positive:$sentence\n";
    }

      [download]
      But, when I run my program, the error message is like this:

      syntax error at calculation.pl line 61, near "} >" Execution of calculation.pl aborted due to compilation errors.

      I do not know how to modify the code, can you help me to solve it. Thank you so much!!!
[reply]
[d/l]
[select]
        Glad to see you have persevered. You need to change the if condition to: 
        if ($probability->{positive} > $probability->{negative} and
    $probability->{positive} > $probability->{neutral}) {
    print "positive: $sentence\n";
}
elsif ($probability->{negative} > $probability->{positive} and
    $probability->{negative} > $probability->{neutral}) {
    print "negative: $sentence\n";
}
elsif ($probability->{neutral} > $probability->{positive} and
    $probability->{neutral} > $probability->{negative}) {
    print "neutral: $sentence\n";
}

        [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom