Re: How do I read in a document, remove the stop words and then write the result to a new file?

You need to be sure to split each line from the input file into words before you check for stopwords. Here's a simplistic example.

...
use Lingua::StopWords qw|getStopWords|;

my $stopwords = getStopWords( 'en' );

open my $infile, '<', 'fulltext.txt' or die "$!\n";
open my $outfile, '>', 'nostopwords.txt' or die "$!\n";

while (my $line = <$infile>) {
    my @words_all = split /\s+/, $line;
    my @words_nostop = grep { !$stopwords->{$_} } @words_all;
    print {$outfile} join( ' ', @words_nostop ), "\n";
}

close $infile;
close $outfile;
[download]

Comment on Re: How do I read in a document, remove the stop words and then write the result to a new file? Download Code