Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Re: Re: Junk NOT words

by kshay (Beadle)
on Oct 31, 2002 at 16:25 UTC ( #209427=note: print w/replies, xml ) Need Help??

in reply to Re: Junk NOT words
in thread Junk NOT words

I wonder if you' d get better results by building a Markov chain based not on the frequency with which a letter follows another individual letter, but on the frequency with which a letter follows a given pair of letters.

I recently did something like this for a web application that goes the other direction: instead of trying to recognize letter combinations as "words," it generates "words" that "make sense" phonetically based on a given data set—in this case, lists of names. You can play with it here. Try a couple of different categories and you'll see that the resulting made-up names are quite phonetically distinctive; they really do seem French or Shakespearean (and within that, male or female) depending on what category you select.

I got the basic approach from this page by Chris Pound. I can't post the code because I wrote it for my employer, but basically I took each list of names, calculated the frequencies, and generated a Perl library containing a long series of assignments like this:

@{$pairs{'ri'}}{qw(a c e n s u)} = (3, 1, 1, 3, 1, 1);

That means that within this data set, "ri" is followed by "a" three times, "c" one time, etc.

Anyway, I'd be curious how this approach might work for word recognition, and how the results for "pair-letter" frequencies might differ from those obtained with "letter-letter" frequencies.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://209427]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (1)
As of 2022-07-01 23:43 GMT
Find Nodes?
    Voting Booth?
    My most frequent journeys are powered by:

    Results (102 votes). Check out past polls.