in reply to Similarity measurement
It really depends on the objective. You could count the occurrence of each word and compare frequencies. They could be identical but the text files could be completely different. A single 'not' could change the meaning of a text into its opposite. The 'diff' tool is useful to compare text files but mainly based on lines. You could consider your text file as a sequence of words and then apply a longest common subsequence algorithm as in Algorithm::Diff.
|
|---|