Just a couple additional points that you might find useful:

Keep your stop words in a separate text file, and read from that file to build up the regex as kennethk suggested above (this way, the stop-word list can be maintained separately from the program code): 

open( STOPWORDS, '<', 'stopword.list' );
my @stopwords = <STOPWORDS>;
chomp @stopwords;

my $stopregex = join '|', map qr/\b\Q$_\E\b/, @stopwords;

[download]
You have a lot of substitutions being done on a word-by-word basis (after splitting each line on whitespace); while the "optimization" value is probably not significant (unless you get into a very large collection of input text), the coding would be much simpler using tr/// on the lines, then splitting to get the tokens: 
my %freq;

while (<FILE>) {
    s/<.+?>/ /g;   # replace tags with spaces
    tr/A-Z0-9?!.,:;()*"`'-/a-z /s;  # convert upper- to lower-case, an
+d
                                 # also convert digits, punct to space

# NOTE: check your output to see whether any other punctuation or
# non-word characters are getting through, and add those to the tr///
# as needed; also: hyphens might need to be treated differently from
# other punctuation (keep as-is, or delete, instead of converting to s
+pace?)

    s/$stopregex//g;  # remove any/all stop words

# at this point, line should contain only word tokens, but 
# use grep, just in case:

    for my $token (  grep /[a-z]/, split ) {  # only count tokens with
+ letters
        $freq{$token}++;
    }
}

[download]
Last thing: I don't know if you intended it, but one of the quote characters in the OP code (that is, one of the marks being removed by s///) was apparently a non-ASCII character (U+201D, "right double quotation mark"). If you really are putting utf8 characters in your code, you may need to include use utf8; If your data is utf8 text, you may need to set utf8 mode in the open statement: open( FILE, '<:utf8', $filename )

In reply to Re: how to create word list from input text file by graff
in thread read whole file in a directory by ask91

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.