in reply to extracting search term context

It's harder to describe without code than with.

First define what constitutes a word. Upper and lower case letters obviously, but what about the quote in "don't"?

Then define what comes in between 'words'. Spaces for sure, but what about punctuation like ,.?!- etc.

Once you've defined your words and separators, you want to capture n words and separators before the match and n separators and words after.

$word = qr[[a-zA-Z']+];; $sep = qr[[ \t.,!?]+];; $n=10; $match = 'match'; $re = qr[((?:$word$sep){$n}\Q$match\E(?:$sep$word){$n})];; print $para =~ $re;; want to extract n words on either side of the match. I am sure modules + such as Plucene or KinoSearch already

So far, so good, but what happens if the word is less that n from the start or end? It won't match, so the quantifier has to become {0,$n}:

$re = qr[((?:$word$sep){0,$n}\Q$match\E(?:$sep$word){0,$n})];; $match = 'CPAN';; print $para =~ m[$re];; There probably is already a CPAN module for this, but I am stuck tryin +g to figure

Better, but it will still fail if the definition of words and separators isn't good enough

For example, if there are numbers in there somewhere. Or brackets. Or quotes. And that's before you get to trying to deal with unicode.

At best the above is a starting point. How far you need to go beyond that will depend upon your application.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

Replies are listed 'Best First'.
Re^2: extracting search term context
by aquarium (Curate) on Jun 09, 2010 at 06:19 UTC
    probably a little better would be to use \b and \w instead of defining a word and separator from sratch, or even use the posix equivalents for more clarity. also a further suggestion to the original question, an alternative would be to read the file in the desired chunks/units to start with, each time through the loop reading one extra word from the file into this comparison buffer and dropping one word off at the end of the buffer. In other words a sliding window/buffer technique with word being step size.
    the hardest line to type correctly is: stty erase ^H
      probably a little better would be to use \b and \w instead of defining a word and separator from sratch

      I invite you to demonstrate this.

      reading one extra word from the file

      How do you read a "word" from a file?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.