Re^4: Text Analysis Tools to compare Slinker and Stinker?

'Misspellings' are precisely where Bayesian filtering, once trained, will help tremendously (though as others have pointed out, never conclusively).

As an example from the anti-spam efforts, once Bayesian filtering was enabled they were amazed that single token with the highest probability of indicating spam was 'FF0000', the hex value for bright red. Unexpected, but damning.

Consistently misspellt words could show up accordingly.

Mattt

Comment on Re^4: Text Analysis Tools to compare Slinker and Stinker?

Replies are listed 'Best First'.
Re: Re^4: Text Analysis Tools to compare Slinker and Stinker? by castaway (Parson) on Jan 22, 2003 at 08:23 UTC
Hmm, I don't know about the rest of you, but when I'm typing a lot, I generally don't make the same mistakes. Ok, when someone consistently misspells the same word, its probably because they don't know how to spell it, but what I'm getting at is, that I more often than not just hit the keys in the wrong order. When there's a lot going on in a forum (or Mud, whatever) then one tends to get out as much as possible in order to keep up, which produces a lot of inconsistent misspellings. (It does in my case anyway, I'd be glad to prove it). Which makes me think that comparing misspellings is not such a good way to do it. Example: German word 'erzaehl', comes out as 'erzahle', 'erzeahl', 'erzaelh' etc. (I've even made aliases for most of these ;) C.	[reply]

Replies are listed 'Best First'.

Re: Re^4: Text Analysis Tools to compare Slinker and Stinker?
by castaway (Parson) on Jan 22, 2003 at 08:23 UTC

Example: German word 'erzaehl', comes out as 'erzahle', 'erzeahl', 'erzaelh' etc. (I've even made aliases for most of these ;)

[reply]