Thinking about the problem in the light of day (rather than stupid-o'clock in the morning), I'm guessing I may need to work on a very different implementation.

The actual purpose of the script is of linguistic interest. One of the ways of generating text that seems like English, without being it, is to generate strings of characters where the next character is selected based on the probability of it appearing in text given the previous n characters. The code I'm using to store probabilities effectively reads through the text and increments the hash entry for each substring of n characters.

So, looking at it in the harsh light of morning, and thinking about what I am actually saying (2^40, which is indeed referring to a 5 character string, is a very large number), I'm probably going to have to find a new way to deal with this...

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.