I am guessing it's because it reads one line at a time from the file.

My guess is different: you've got several nested loops - for each word, you loop over @excluded and @excluded_chars and run a regex for each, so with what you showed here, that's 50 regex runs per word. Instead, see Building Regex Alternations Dynamically. Also, I use lc to lowercase instead of a regex, which handles unicode as well (though you would need to make sure your inputs are properly decoded for that case).

#!/usr/bin/env perl
use warnings;
use strict;
use autodie;

my @excluded_words  = qw( a about although also an and another are as
    at be been before between but by can do during for from has how
    however in into is it many may more most etc );
my @excluded_chars = ( "'", ':', '@', '-', '~', ',', '.', '(', ')',
    '?', '*', '%', '/', '[', ']', '=', '"' );
my ($word_regex) = map {qr/$_/} join '|', map {"\\b".quotemeta."\\b"}
    sort { length $b <=> length $a or $a cmp $b } @excluded_words;
my ($char_regex) = map {qr/$_/} join '|', map {quotemeta}
    sort { length $b <=> length $a or $a cmp $b } @excluded_chars;

my %count;
while (<>) {
    for (split) {
        $_ = lc;
        s/$char_regex//g;
        s/$word_regex//g;
        $count{$_}++;
    }
}

for my $word ( sort { $count{$a} <=> $count{$b}
    or $a cmp $b } keys %count ) {
        print "$count{$word} $word\n";
}
[download]

On my machine on a test file, this is faster than the original code by a factor of roughly 7x.

Update: The first regex is written a bit more cleanly as: my ($word_regex) = map {qr/\b(?:$_)\b/} join '|', map {quotemeta} sort ...

Ok that is cool, I didn't know about 'map' and 'quotemeta'. I think your code runs about the same speed as the shell script.

Try this. It does the lc() and the tr/// only once.

#!/usr/bin/perl

use strict; # https://perlmonks.org/?node_id=11116620
use warnings;

my @excluded = qw( a about although also an and another are as at be b
+een
    before between but by can do during for from has how however in in
+to is
    it many may more most etc );

local $/;
my %count;
$count{$_}++ for split ' ', (lc <>) =~ tr!-'@~,.()?*%/[]="!!dr;
delete @count{@excluded};

print "$count{$_} $_\n"
  for sort { $count{ $b } <=> $count{ $a } || $a cmp $b } keys %count;
[download]

Independently I have arrived at a similar solution.

#!/usr/bin/env perl 
use strict;
use warnings;

my %excluded = map { $_ => 1 }
    qw( a about although also an and another are as at be been
    before between but by can do during for from has how however in in
+to is
    it many may more most etc );
my %count;

{
    local $/ = "";
    while (<>) {
        tr {A-Z':@~,.()?*%/[]="-}{a-z}d;
        foreach (split) {
            $count{$_}++ unless $excluded{$_};
        }
    }
}

foreach my $word (sort { $count{$a} <=> $count{$b} or $a cmp $b } keys
+ %count) {
    print "$count{$word} $word\n";
}
[download]

I've leveraged the requirement of only lowercasing the ascii letters by incorporating it into the tr/// and I've gone for paragraph mode instead of a single slurp, just in case :-)

Both solutions run in similar times and about 100x faster than the original code:

$ time ./11116620.pl < Frankenstein.txt > orig.out

real    0m9.381s
user    0m9.366s
sys    0m0.007s
$ time ./wordcount.pl < Frankenstein.txt > hippo.out

real    0m0.089s
user    0m0.081s
sys    0m0.008s
$ time ./tybalt.pl < Frankenstein.txt > tybalt.out

real    0m0.090s
user    0m0.084s
sys    0m0.005s
[download]

There are some minor differences between all three outputs but without a tighter spec these aren't overly concerning.

I don't understand your "list of excluded characters". In the bash version you are using tr which only works on characters but in the Perl version your list uses two character strings instead of single characters. Also, Perl has a tr/// operator which operates pretty much the same as the command line tool (on single characters).

Your "list of excluded words" would probably be better as a hash.

Reading a file a line at a time shouldn't be a problem as Perl IO is buffered, however, you can change how much data is read in by changing the Input Record Separator ($/) to read in a whole file.

The comment in your program says "# remove non-letter characters", however you are only removing certain punctuation characters.

Perhaps you want something like this:

#!/usr/bin/perl
# word counting program
use strict;
use warnings;
use autodie;

# list of excluded words
my %excluded = map { $_ => 1 } qw( a about although also an and anothe
+r are as at be been
    before between but by can do during for from has how however in in
+to is
    it many may more most etc );

## list of excluded characters
#my @excluded_chars = ( "\\'", "\\:", "\\@", "\\-", "\\~", "\\,", "\\.
+", "\\(",
#    "\\)", "\\?", "\\*", "\\%", "\\/", "\\[", "\\]", "\\=", '"'
#    );

# Read in whole file (uncomment next line)
# local $/;

my %count;  # this will contain many words

while (<>) {
    # remove punctuation
    tr{':@\-~,.()*%/[]="}{}d;
    foreach ( split ) {
        # remove excluded words
        next if exists $excluded{ lc() };

        ++$count{ lc() };  # count each separate word
    }
}

foreach my $word (sort { $count{ $a } <=> $count{ $b } || $a cmp $b } 
+keys %count) {
    print "$count{$word} $word\n";
}
[download]

Beyond that, suggestions below about improving your algorithm and utlizing Perl builtins seem to suffice.