millsperl has asked for the wisdom of the Perl Monks concerning the following question:

Need to exclude non ascii characters, i.e. greater than 7F(hex). Currently the perl script excludes a couple of characters, such as null. This is done by declaring the character to be excluded, i.e. $remove_char = chr(hex("00")); Then each line in the file is read and the null replaced by space, i.e. $myline =~ s/$remove_char/ /g; How should I code the declaration and statement that does the substitution?

Replies are listed 'Best First'.
Re: Excluding non ascii characters
by jwkrahn (Abbot) on Apr 06, 2009 at 19:02 UTC

    Perhaps you want something like this: 

    $myline =~ s/[^[:ascii:]]/ /g;

    [download]
[reply]
[d/l]
Re: Excluding non ascii characters
by linuxer (Curate) on Apr 06, 2009 at 19:07 UTC

    What about a character range? And if you want to delete those characters, you also can use tr///.

    $myline =~ tr/\x80-\xff//d;

    [download]

    or if you want to stick to s///. 

    $myline =~ s/[\x80-\xff]/ /g;

    [download]
[reply]
[d/l]
[select]
      That will only remove the non-ASCII characters that are in iso-latin-1, a very small portion of non-ASCII characters. Fix: 
      # Replace with space
$myline =~ s/[^\x00-\x7F]/ /g;

# Delete character
$myline =~ tr/\x00-\x7F//cd;
$myline =~ s/[^\x00-\x7F]//g;

      [download]
[reply]
[d/l]
[select]
Re: Excluding non ascii characters
by ikegami (Patriarch) on Apr 06, 2009 at 20:52 UTC
    Here's a solution that works with arbitrary encodings rather than just ASCII: 
    use Encode qw( decode encode );

my $input    = '...';
my $enc      = 'US-ASCII';
my $spc      = encode($enc, ' ');
my $filtered = decode($enc, encode($enc, $s, sub { $spc }));

    [download]

    Encode

[reply]
[d/l]
Re: Excluding non ascii characters
by ikegami (Patriarch) on Apr 06, 2009 at 20:55 UTC
[reply]
      The author hasn't updated it in over 10 years, and refuses to respond to bug reports, patches, or even accepting comaintainers.
[reply]

Back to Seekers of Perl Wisdom