in reply to Re: Interleaving bytes in a string quickly
in thread Interleaving bytes in a string quickly

Were you looking for a pure perl or an inline solution and which one are you going to use?

Cheers Rolf

  • Comment on Re^2: Interleaving bytes in a string quickly

Replies are listed 'Best First'.
Re^3: Interleaving bytes in a string quickly
by BrowserUk (Patriarch) on Feb 26, 2010 at 17:02 UTC

    The problem arose from Re^3: encoding hdmi video byte. How to efficiently interleave the constant bytes into the tr/// encoded 100MB buffer loads. So I was looking for a pure Perl solution.

    If it was for me, I'd probably do the whole thing in C.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Hmm ...

      nowadays graphic cards can do these kind of transformations at lightspeed, I would try to find drivers which can be used from perl

      My next guess is assembler, the opcodes needed are very processor independent...

      Cheers Rolf

        graphic cards can do these kind of transformations at lightspeed,

        Yes. But all the APIs for getting at the power of GPUs that I've looked at are horribly complex.

        Given that the following can encode 3GB (about 2 1/2 hours of HD video?), in 3 minutes 20 seconds, maybe I wouldn't even bother with C. Most of the time is spent reading and writing the disk.

        #! perl -slw use strict; use Time::HiRes qw[ time ]; use constant BUFSIZ => 4*1024; #50 * 1024**2; my $start = time; my $mask = ( chr( 0 ) . chr( 3 ) ) x BUFSIZ; my %map; for my $i ( 0 .. 255 ) { my @Q; my @D = split '', unpack 'b8', chr $i; $Q[ 0 ] = $D[ 0 ]; $Q[ 1 ] = $D[ 1 ] ^ $Q[ 0 ]; $Q[ 2 ] = $D[ 2 ] ^ $Q[ 1 ]; $Q[ 3 ] = $D[ 3 ] ^ $Q[ 2 ]; $Q[ 4 ] = $D[ 4 ] ^ $Q[ 3 ]; $Q[ 5 ] = $D[ 5 ] ^ $Q[ 4 ]; $Q[ 6 ] = $D[ 6 ] ^ $Q[ 5 ]; $Q[ 7 ] = $D[ 7 ] ^ $Q[ 6 ]; my $outb = ord pack 'b8', join'',@Q; $map{ $i } = $outb; } my $target = quotemeta pack 'C256', map $map{ $_ }, 0 .. 255; eval "sub encodeHDMI { tr[\x00-\xff][$target]; }"; warn "$@" if $@; $/ = \BUFSIZ; open IN, '<:raw', $ARGV[ 0 ] or die "$ARGV[ 0 ] : $!\n"; binmode STDOUT, ':raw'; while( <IN> ) { encodeHDMI(); my $out = pack 'S*', unpack 'C*', $_; $out |= $mask; # print; } printf "File: %d bytes took %.2f seconds\n", -s( $ARGV[0] ), time() -$start; __END__ C:\test>825104 syssort File: 3160000000 bytes took 200.48 seconds

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.