in reply to Interleaving bytes in a string quickly

I declare lanX the winner (with jmacnamara a close second) for a pure perl solution.

fletch's Inline C for absolute speed.

#! perl -slw use 5.010; use strict; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'END_C', NAME => '_825494', CLEAN_AFTER_BUILD => 0; SV *interleave( SV *in, char other ) { size_t len = SvCUR( in ), i; SV *out = newSV( sizeof(char) * len * 2 ); char *outp = SvPVX( out ); char *p = SvPVX( in ); for( i = 0; i < len; ++i ) { *outp++ = *p++; *outp++ = other; } *outp = '\0'; SvPOK_on( out ); SvCUR_set( out, len * 2 ); return out; } END_C use Devel::Peek; use Benchmark qw[ cmpthese ]; use constant DEBUG => 0; ## Set to 1 to see first 20 bytes of results; our $in = chr(1) x 1e6; our $mask = ( chr( 0 ) . chr( 64 ) ) x 1e6; cmpthese DEBUG ? 1 : -3, { buk => q[ my $out = join( chr(64), unpack '(A1)*', $in ) . chr(64); print 'buk ', unpack 'C20', $out if DEBUG; ], shmem => q[ my $out = chr(64) x ( length( $in ) * 2 ); vec( $out, $_<<1, 8 ) = vec( $in, $_, 8 ) for 0..length( $in ) + -1; print 'shmem ', unpack 'C20', $out if DEBUG; ], salva => q[ my $out = chr(64) x ( length( $in ) * 2 ); substr( $out, $_ * 2, 1, substr( $in, $_, 1 ) ) for 0..length( + $in )-1; print 'salva ', unpack 'C20', $out if DEBUG; ], lanX => q[ my $out = pack 'S*', unpack 'C*', $in; $out |= $mask; print 'lanX ', unpack 'C20', $out if DEBUG;; ], jmac => q[ my $hi_bytes = pack 'v*', unpack 'C*', $in; my $lo_bytes = pack 'n*', ( 64 ) x length $in; my $out = $hi_bytes | $lo_bytes; print 'jmac ', unpack 'C20', $out if DEBUG;; ], fletch => q[ my $out = interleave( $in, chr( 64 ) ); print 'fletch', unpack 'C20', $out if DEBUG;; ], }; __END__ C:\test>825494 buk 164164164164164164164164164164 (warning: too few iterations for a reliable count) fletch164164164164164164164164164164 (warning: too few iterations for a reliable count) jmac 164164164164164164164164164164 (warning: too few iterations for a reliable count) lanX 164164164164164164164164164164 (warning: too few iterations for a reliable count) salva 164164164164164164164164164164 (warning: too few iterations for a reliable count) shmem 164164164164164164164164164164 (warning: too few iterations for a reliable count) C:\test>825494 Rate buk shmem salva jmac lanX fletch buk 2.10/s -- -26% -37% -76% -81% -99% shmem 2.84/s 36% -- -14% -68% -74% -99% salva 3.32/s 58% 17% -- -62% -69% -99% jmac 8.84/s 321% 211% 166% -- -18% -97% lanX 10.8/s 415% 279% 225% 22% -- -97% fletch 334/s 15849% 11662% 9968% 3685% 3000% --

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"I'd rather go naked than blow up my ass"

Replies are listed 'Best First'.
Re^2: Interleaving bytes in a string quickly
by ikegami (Patriarch) on Feb 26, 2010 at 18:51 UTC
    There's a few problems with your C code.
    • It can segfault instead of stringifying (interleave(123, "\x40")).
    • It can silently encode your bytes using UTF-8.
    • Magic isn't handled if any is present.

    All of these are solved by using SvPVbyte instead of SvPVX.

      That's okay. It survived it's intended lifecycle(*) without incident.

      (*)The benchmark.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        What's the point of benchmarking code you can't use?
Re^2: Interleaving bytes in a string quickly
by LanX (Saint) on Feb 26, 2010 at 16:38 UTC
    Were you looking for a pure perl or an inline solution and which one are you going to use?

    Cheers Rolf

      The problem arose from Re^3: encoding hdmi video byte. How to efficiently interleave the constant bytes into the tr/// encoded 100MB buffer loads. So I was looking for a pure Perl solution.

      If it was for me, I'd probably do the whole thing in C.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        Hmm ...

        nowadays graphic cards can do these kind of transformations at lightspeed, I would try to find drivers which can be used from perl

        My next guess is assembler, the opcodes needed are very processor independent...

        Cheers Rolf