Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

You make some very worthwhile observations. From my perspective, the extensive similarities of the various texts is what provides the incentive to identify them. If the text were much more diverse, it would be difficult to find comparisons with a sufficient similarity index to pass the threshold, at which point the results would lose significance.

Any algorithm that gives less weight to the more common words will, in my opinion, be less able to handle this particular corpus and yield quality results. It is true that many of the keywords are used ubiquitously throughout the Bible: words like Lord, God, father, son, praise, faith, etc. There are also words that are much more common than in other types of text, such as inherit, blessing, perpetual, covenant, sacrifice, etc. And then, as has been pointed out, many of the words are of old-English form and would not be easy to accommodate with a stemmer: verb forms like hadst, hast, hath, gavest, giveth, rebuketh, etc.; pronouns like thee, thou, thine, and ye; and words like hence, hither, whither, whence, lest, whatsoever, whomsoever, etc. To add a little spice, there are the more unusual vocabulary words, like apothecary, winnowing, and all the wonderful names like Belteshazzar, Nebuchadnezzar, Abaddon, Apollyon, Zaphnathpaaneah, Mahershalalhashbaz, Sennacherib, etc. There are some interesting features of the language, along with some of the more repetitious aspects.

The entire corpus contains close to 13,000 words--not far from the average vocabulary of a high-school graduate these days.

A list of all the words in the KJV Bible, showing the number of occurrences for each, can be found HERE. It helps one to understand the peculiarities of this particular lexicon.



In reply to Re^6: String Comparison & Equivalence Challenge by Polyglot
in thread String Comparison & Equivalence Challenge by Polyglot

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or or How to display code and escape characters are good places to start.
Log In?

What's my password?
Create A New User
Domain Nodelet?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (5)
As of 2022-05-23 23:45 GMT
Find Nodes?
    Voting Booth?
    Do you prefer to work remotely?

    Results (82 votes). Check out past polls.