doonyakka has asked for the wisdom of the Perl Monks concerning the following question:

Hi :) I'm a complete newbie at Perl (having never done any programming at all before), but I've been having a lot of fun hacking away for the past coupla weeks. I've been trying to write some code that will scan HTML files for particular words, and then output the full sentences those words are in.

I've managed to get something that works perfectly - so far ;) - but, as I want it to scan multiple languages, I've tried to specify locales. I've followed instructions for using 'locale', but it hasn't worked, so I've had to specify accented words in the regex.

Here are a few excerpts from what I've done so far (I've 'elided' the irrelevant bits with ...):
#!/usr/bin/perl use locale; use POSIX(locale_h); # query and save the old locale $old_locale = setlocale(LC_CTYPE); # set locale (doesn't work) setlocale(LC_CTYPE, "ISO8859"); print "Input word: "; $word = <STDIN>; chomp($word); ... $fileNo = 1; # HTML filenames have already been converted to numbers ( +eg., 1.html etc) while ($fileNo <= 610) { open FH, "K:\\$fileNo.html" or die "Can't open: $!"; open OUT, ">>K:\\sentences.txt"; # print OUT "$fileNo.html\n"; # print "$fileNo.html\n"; $/ = "."; while (<FH>) { ... # if (/(^|\. ])([a-z0-9\,\:\;\"\' ]* $word[a-z0-9\,\:\;\"\'\. ] +*)$/i) { # this one doesn't work! if (/(^|\. ])([a-záéíóúñü0-9\,\:\;\"\' ]* $word[a-záéíóúñü0-9\ +,\:\;\"\'\. ]*)$/i) { # so I've had to do this instead! print OUT "\n$fileNo.html: $2\n"; print "\n$fileNo.html: $2\n"; } } close OUT; close FH; $fileNo++; }
What am I doing wrong? I've downloaded the locale specifications and have tried es_ES, fr_FR etc, but still no luck. Excuse me if I've made a really elementary goof, but it's all been trial and error so far. Thanks, this seems like a great place to learn and share wisdom!
Best regs,
DoonYakka

Replies are listed 'Best First'.
Re: problems with locale
by chromatic (Archbishop) on Jul 09, 2001 at 02:12 UTC
    Locales affect the character classes in regular expressions, not explicitly named characters. If you can get by with specifying \w and some additional punctuation, it should work better.

    Disclaimer: I'm not as smart as I think I am, and I only got a third of the way through perllocale before Unicode-malaise struck yet again.

Re: problems with locale
by wog (Curate) on Jul 09, 2001 at 02:12 UTC
    The locale pragma does not change the behavior of character classes like [a-z0-9,:;"' ] in regular expressions. If you want to get the accents as expected you should be able to use something like [[:lower:]0-9:;"' ] instead. These are described in perlre. Search for "POSIX character class".

    update: I should clarify that POSIX character classes aren't the only way locales are supported in regex's character classes, of course. As chromatic says, the reason that yours didn't work is because you enumerated a-z explictly. As long as you aren't using explict ranges like that including accented characters, etc. won't be a problem; things like \w work as expected under use locale.

    (Why aren't you using or die ... on the second open there? Also, perl lets you use / instead of \\ for things like open. )

    Please turn on perl's features to help you.