in reply to testing if a string is ascii

How about using transliteration? Transliteration is marginally quicker than pattern matching. Here's a solution that will trigger the error condition if the string contains anything outside of \0-\x7f range. It works by using the /c modifier on a tr/// transliteration operator. Refresher course: the /c modifier complements the search list, and if no "replace" list is specified, one that exactly matches the search list is generated behind the scenes. That has the effect of leaving the original string untouched, only counting characters that match the criteria (or in this case, counting the ones that match the complement to the criteria specified, thanks to /c)

for my $str ( "\x7f", "asdf", "asdf\x8f", "\x8f" ) { print "$str contans ", ( $str =~ tr/\0-\x7f//c ) ? "non-" : "only ", "ascii.\n"; }

A lot of code there is just testing framework. The engine at work is this:

tr/\0-\x7f//c

If that tests positive, you've got non-ascii characters in your string. Details about tr/// can be found in perlop.


Dave

Replies are listed 'Best First'.
Re^2: testing if a string is ascii
by tye (Sage) on Sep 04, 2006 at 14:20 UTC
    How about using transliteration? Transliteration is marginally quicker than pattern matching.

    I think your dogma got hit by some carma. How do you think using tr/// to hit every single character in a string will be faster than m// (without /g) that can immediately stop when it finds the first problem character?

    - tye