I might add that in human language \w+ is usually both too broad (because it contains the underscore _ and digits, which usually don't appear in words in human language) and too narrow (don't), and in general word recognition is language dependent.

If you try to grab words with a regex, the table of Unicode properties in perlunicode can be very helpful. For example in many languages words consist of letters which can be matched with \pL or \p{Letter}, and marks (\pM or \p{Mark}).

(Marks are combining characters that can modify letters or other characters. A well-known example is the "combining grave accent", which turns an A into a À)

Update: To clarify this further: in this context the one thing that Unicode buys you is that you don't have to enumerate characters to build your character classes. That's a cumbersome task, and usually done wrong because there's a huge set of characters. It doesn't help you with your mental decision of what you consider a word.

In reply to Re: What Is A Word? by moritz
in thread What Is A Word? by Limbic~Region

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.