in reply to OT (for now): Mis-spelling research

 Whilst I'm not aware of any real research I guess this is closely related to some work that I've been doing.

 After reading about bayasian filters for a while, and being impressed that they work so well I started thinking of other problems that could be solved statistically.

 One thing that I often do is misspell particular words, which don't get caught because I'll use the wrong word - like using "they" instead of "this". (Amazing how often I do that).

 Another class of errors is the holding of the shift key for too long. This resulted in the previous sentence starting "ANother...", and results in frequent uses of "THe", "LIvejournal", etc.

 The first problem I've not solved, but the second can be detected and corrected if you look at frequency analysis of letter pairs.

 I've written code that sums up the changes of a given letter being followed by another given letter. So for example the chance of "q" being followed by "u" is 95%. The chance of "T" to be followed, legitimately, by "H" is 7%.

 With a big enough sample I can flag errors with 98% accuracy - without using a dictionary.

 Maybe this is a cool use for perl?

Steve
---
steve.org.uk
  • Comment on Re: OT (for now): Mis-spelling research

Replies are listed 'Best First'.
Re: Re: OT (for now): Mis-spelling research
by BrowserUk (Patriarch) on Oct 15, 2003 at 13:14 UTC

    I'm not actually looking at how to correct misspellings at this point, though I agree that the use of Bayasian filters for this makes a lot of sense and is a very cool use of perl.

    My interest stems from my noticing that the frequency and range of the words I misspell increases dramatically when I am trying to write or type something I feel passionate about and/or have a large volume of information to convey.

    It seems to me that my brain gets ahead my fingers abilities to type the stuff and I find that I will sometime contract two similar words, that may be several words apart in the sentence am trying to type, into a single word and completely omit the intervening words. Another common occurance in my own typing is when I make a typo in a word that I normally spell correctly, notice and go back to correct it and get a mental block about the spelling.

    An example of this occured whilst type the word "passionate" above. I ommited one of the S's, and interchanged the I & O: "pasoinate". Whilst it was perfectly obvious to me that it was misspelt, for a few seconds, I simply could not see how to correct it. It required me to stop thinking about what I was going to type and concentrate specifically upon that word before I could see the correction.

    I've been trying to think of a way of measuring this phenomena -- if that's what it is. I keep thinking about some sort of program to try and apply pressure to the subject typing (me:) and also some way to force me to try and type one pice of information whilst thinking about another, but I thought I might find some research and possibly some test methods out there somewhere, but they have escaped me so far.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "Think for yourself!" - Abigail
    Hooray!