text mining text tokenizing

onyourmark has asked for the wisdom of the Perl Monks concerning the following question:

Hello. Can anyone recommend the best module for taking text and putting it into a term-frequency matrix? And is there any module that has text mining functions? Thanks

Comment on text mining text tokenizing

Replies are listed 'Best First'.
Re: text mining text tokenizing by planetscape (Chancellor) on Dec 23, 2010 at 08:06 UTC
Perhaps you may want to take a look at Ted Pedersen's Ngram Statistics and SenseClusters packages. HTH, planetscape	[reply]
Re: text mining text tokenizing by Anonymous Monk on Dec 22, 2010 at 13:39 UTC
Can you find a single module that does that? If you can find one, its automatically the best one :)	[reply]
Re^2: text mining text tokenizing by onyourmark (Initiate) on Dec 22, 2010 at 14:36 UTC
No, but I am new to the perl modules.	[reply]
Re^3: text mining text tokenizing by Anonymous Monk on Dec 22, 2010 at 14:49 UTC
Sorry, I'm not a linguist. These might be of interest Lingua::TreeTagger Lingua::Wordnet Lingua::EN::Ngram Text::Positional::Ngram KinoSearch	[reply]