Re: Answer: How do I count the frequency of words in a file and save them for later?

There have been - and probably will be - quite a few posts regarding word counts. This solution doesn't work. To give just one example, "can't" ends up as 1 "can" and 1 "t". Other solutions often have it as "cant", but what is really needed is testing to see if the apostrophe has at least one letter on each side. Also, what about end of line word splits? A word like:

google-
plex

Should be converted to googleplex before counting. I imagine there are one or two other things to program in as well.

I'm not saying this is necessarily a bad place to start, but you need to program in some modifications. Better get cracking.

Comment on Re: Answer: How do I count the frequency of words in a file and save them for later?

Replies are listed 'Best First'.
Re^2: Answer: How do I count the frequency of words in a file and save them for later? by planetscape (Chancellor) on Jun 13, 2005 at 06:54 UTC
Thanks for pointing that out, TedPride. I have only used this for relatively simple things. Normally, for my purposes, I use Ted Pedersen's Ngram Statistics Package. planetscape	[reply]