chuckd has asked for the wisdom of the Perl Monks concerning the following question:

Hi, Is it possible to use more than one character for a transliteration (tr///)? I want to replace '&' with 'amp' and I don't wont to use s/// I guess my question is, is it possible to translate strings without using the substitute operator? I'm trying to find a faster method than s/// because I have many things to substitute over very long strings. I'm trying to find a way to sped things up. I would like to speed this up by putting them all in one reg exp so that I don't have to walk through the long string three times:
$string =~ s/\&/amp/g;
$string =~ s/\>/gt/g;
$string =~ s/\</lt/g;

Replies are listed 'Best First'.
Re: using tr///
by BrowserUk (Patriarch) on Aug 01, 2008 at 04:51 UTC

    If you're using 5.8.*, then stick with what you have:

    c:\test>\perl\bin\perl5.8.6.exe -s junk9.pl -N=1e2 Rate ysth buk tye OP ysth 4371/s -- -47% -49% -75% buk 8269/s 89% -- -3% -53% tye 8542/s 95% 3% -- -52% OP 17779/s 307% 115% 108% -- c:\test>\perl\bin\perl5.8.6.exe -s junk9.pl -N=1e5 Rate ysth buk tye OP ysth 3.75/s -- -36% -39% -64% buk 5.88/s 57% -- -4% -44% tye 6.15/s 64% 5% -- -41% OP 10.5/s 179% 78% 70% --

    If you're using 5.10, as unintuative as it sounds, a split/join solution seems to be the fastest. Especially when the strings get up into the megabyte range:

    c:\test>\Perl510\bin\perl5.10.0.exe -s junk9.pl -N=1e2 Rate ysth tye OP buk ysth 2326/s -- -50% -64% -72% tye 4670/s 101% -- -27% -45% OP 6417/s 176% 37% -- -24% buk 8419/s 262% 80% 31% -- c:\test>\Perl510\bin\perl5.10.0.exe -s junk9.pl -N=1e5 Rate ysth OP tye buk ysth 1.83/s -- -32% -46% -69% OP 2.70/s 48% -- -21% -55% tye 3.42/s 87% 26% -- -43% buk 5.98/s 227% 121% 75% -- c:\test>\Perl510\bin\perl5.10.0.exe -s junk9.pl -N=1e6 s/iter OP ysth tye buk OP 43.0 -- -54% -60% -96% ysth 19.9 116% -- -14% -91% tye 17.1 151% 16% -- -90% buk 1.72 2400% 1056% 896% --

    I'm quite prepared to have my intuition confirmed and benchmark proven wrong, but this is the second time this year I've benchmarked this and I can't see anything wrong with my benchmark.

    The thing to note is that the performance of the split/join solution (buk) hardly varies between the two builds, but the cost of the substitutions seems to have really increased in 5.10. With or without capturing parens or /e.

    The benchmark code:

    #! perl -slw use strict; use Benchmark qw[ cmpthese ]; my %subst = ( '&' => 'amp', '>' => 'gt', '<' => 'lt' ); our $ysth = join('|', map quotemeta, sort { length($b) <=> length($a) +|| $b cmp $a } keys %subst); our $N ||= 1e3; our $data = join( '', map{ 'junk ' . (qw[ & < > ])[ rand 3 ] } 1 .. $ +N ); cmpthese -3, { OP => sub { $_ = $data; s{\&}{amp}g; s{\<}{lt}g; s{\>}{gt}g }, tye => sub { $_ = $data; s{([&<>])}{$subst{$1}}ge; }, ysth => sub { $_ = $data; s{($ysth)}{$subst{$1}}ge; }, buk => sub { $_ = $data . ' '; $_ = join 'amp', split '&'; $_ = join 'lt', split '<'; $_ = join 'gt', split '>'; chop; }, };

    Have at it!


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: using tr/// (hash)
by tye (Sage) on Aug 01, 2008 at 03:38 UTC
    my %r = qw( & amp > gt < lt ); $string =~ s/([&<>])/$r{$1}/g;

    But I seriously doubt any speed difference will really be significant.

    - tye        

Re: using tr///
by ysth (Canon) on Aug 01, 2008 at 03:43 UTC
    A common way to do this for arbitrary literal string replacements would be:
    my %subst = ( '&' => 'amp', '>' => 'gt', '<' => 'lt' ); my $alternatives = join('|', map quotemeta, sort { length($b) <=> leng +th($a) || $b cmp $a } keys %subst); $string =~ s/($alternatives)/$subst{$1}/ge;
    Have you considered HTML::Entities?

    Hmm. I suppose if your input string were just ASCII, you could do tr///'s to carefully selected unicode characters, then turn off the utf8 flag and do a second tr to produce the desired letters. But that would be sick and twisted.