BrowserUk has asked for the wisdom of the Perl Monks concerning the following question:

I have an 2-byte string, an I wish to extract the numeric value of its 4 nybbles into an array. Below I have 5 working variations, but they are all quite complex and quite slow.

print map{hex} (split'', unpack 'h4', $n)[2,3,0,1]; #! Gives >> 15 0 0 15 print map{ ( ord($_) & 0xf0 ) >> 4, ord($_) & 0x0f } split//, $n; #! Gives >> 15 0 0 15 print map{ vec $n, $_, 4 } 0 .. 3; #! Gives >> 0 15 15 0 print +(map{ vec $n, $_, 4 } 0 .. 3)[2,3,0,1]; #! Gives >> 15 0 0 15 print map{ ( unpack( 'n' , $n ) >> ( 4 * $_ ) ) & 0xf } 0 .. 3; #! Gives >> 15 0 0 15 $n = unpack 'n', $n; print map{ ($n >>( 4 * $_ )) & 0xf } 0 .. 3; #! Gives >> 15 0 0 15

There has to be a better way?

Code and benchmark

#! perl -slw use strict; use Benchmark qw[cmpthese]; our $n = "\xf0\x0f"; print unpack 'hhhh', $n; #! Gives >> 0 f print unpack 'h h h h', $n; #! Gives >> 0 f print unpack 'h2 h2', $n; #! Gives >> 0f f0 +# two 2-byte strings print unpack 'h4', $n; #! Gives >> 0ff0 # + a single 4 char string print map{hex} (split'', unpack 'h4', $n)[2,3,0,1]; #! + Gives >> 15 0 0 15 print map{ ( ord($_) & 0xf0 ) >> 4, ord($_) & 0x0f } split//, $n; #! + Gives >> 15 0 0 15 print map{ vec $n, $_, 4 } 0 .. 3; #! + Gives >> 0 15 15 0 print +(map{ vec $n, $_, 4 } 0 .. 3)[2,3,0,1]; #! + Gives >> 15 0 0 15 print map{ ( unpack( 'n' , $n ) >> ( 4 * $_ ) ) & 0xf } 0 .. 3; #! + Gives >> 15 0 0 15 $n = unpack 'n', $n; print map{ ($n >>( 4 * $_ )) & 0xf } 0 .. 3; #! + Gives >> 15 0 0 15 our @nybbles = (); cmpthese( -1, { #! 1 map, 4 hexes (Just my luck!), a split, an unpack, a list and +a slice. map_hex_split_unpack => q[ @nybbles = map{hex} (split'', unpack 'h4', $n)[2,3,0,1]; ], #! 1 map, 2 ords, 2 ands, shift right 4, 1 split and a partridge i +n a pair tree! map_ord_and_shift_split => q[ @nybbles = map{ ( ord($_) & 0xf0 ) >> 4, ord($_) & 0x0f } sp +lit//, $n; ], #! 1 map, 4 vecs, two lists, one splice. Phew! map_vec_lists_splice => q[ @nybbles = map{ ( unpack( 'n' , $n ) >> ( 4 * $_ ) ) & 0xf } + 0 .. 3; ], #! 1map, 4 X ( unpack, shift, mult, & ), a list #! and enough brackets to keep a LISP afficianardo happy for days. map_unpack_shift_mult_and_list => q[ @nybbles = map{ ( unpack( 'n' , $n ) >> ( 4 * $_ ) ) & 0xf } + 0 .. 3; ], #! Factor out the unpack. Slightly better. unpack_map_shift_and_list => q[ $n = unpack 'n', $n; @nybbles = map{ ( $n >> ( 4 * $_ ) ) & +0xf } 0 .. 3; ], }); __DATA__ C:\test>test map_hex_split_unpack: 1 wallclock secs @ 10317.66/s (n=107 +51) map_ord_and_shift_split: 1 wallclock secs @ 7324.69/s (n= 76 +25) map_unpack_shift_mult_and_list: 2 wallclock secs @ 12869.53/s (n=146 +97) map_vec_lists_splice: 1 wallclock secs @ 12954.41/s (n=133 +56) unpack_map_shift_and_list: 1 wallclock secs @ 15328.34/s (n=153 +59) Rate MOASP MHSU MUSMUL MVLS +UMSL map_ord_and_shift_split 7325/s -- -29% -43% -43% +-52% map_hex_split_unpack 10318/s 41% -- -20% 20% +-33% map_unpack_shift_mult_and_list 12870/s 76% 25% -- -1% +-16% map_vec_lists_splice 12954/s 77% 26% 1% -- +-15% unpack_map_shift_and_list 15328/s 109% 49% 19% 18% + -- C:\test>

In anyone has/or can produce a Inline C or XS equivalent built for AS 633 or AS 802, I'd sell my soul er... be very grateful>


Examine what is said, not who speaks.

The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

Replies are listed 'Best First'.
Re: Nybbles take to long (Solved?)
by BrowserUk (Patriarch) on Feb 02, 2003 at 12:33 UTC

    OKay. After walking away for a while the penny dropped. Use vec instead of unpack and avoid map and a slice by unwinding the loop.

    Addition to the benchmark above.

    vec_x4 => q[ @nybbles = (vec($n, 2, 4), vec($n, 3, 4), vec($n, 0, 4), vec +($n, 1, 4)); ],

    Results for comparison with those above

    vec_x4 42230/s 490% 319% 231% 230% 176% --

    Which by my reckoning, (15328*20210)/8188 = 37833 -v- 42230 makes this around 10% faster than the Inline-C version, thought the difference will probably be absorbed once its wrapped in a function as that is. Still, Perl wins again.

    I'd still like to see the XS version.


    Examine what is said, not who speaks.

    The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

      just for kicks...

                                        Rate map_hex_split_unpack map_ord_and_shift_split map_unpack_shift_mult_and_list map_vec_lists_splice unpack_map_shift_and_list vec_x4_sub woot vec_x4_inline
      
      map_hex_split_unpack            4887/s                   --                    -28%                           -33%                 -33%                      -39%       -63% -75%          -77%
      map_ord_and_shift_split         6762/s                  38%                      --                            -8%                  -8%                      -15%       -49% -66%          -69%
      map_unpack_shift_mult_and_list  7314/s                  50%                      8%                             --                  -0%                       -8%       -44% -63%          -66%
      map_vec_lists_splice            7314/s                  50%                      8%                             0%                   --                       -8%       -44% -63%          -66%
      unpack_map_shift_and_list       7952/s                  63%                     18%                             9%                   9%                        --       -40% -60%          -63%
      vec_x4_sub                     13175/s                 170%                     95%                            80%                  80%                       66%         -- -33%          -39%
      woot                           19728/s                 304%                    192%                           170%                 170%                      148%        50%   --           -9%
      vec_x4_inline                  21606/s                 342%                    220%                           195%                 195%                      172%        64%  10%            --
      

      i may be wrong, but for me, Inline::C == XS.

      the generated XS file

      # index_pl_8d47.xs
      #include "EXTERN.h"
      #include "perl.h"
      #include "XSUB.h"
      #include "INLINE.h"
      
      void woot(char *foo) {
              Inline_Stack_Vars;
              Inline_Stack_Reset;
              Inline_Stack_Push(newSViv( (unsigned char)foo[0] >>   4 ));
              Inline_Stack_Push(newSViv( (unsigned char)foo[0] & 0x0f ));
              Inline_Stack_Push(newSViv( (unsigned char)foo1 >>   4 ));
              Inline_Stack_Push(newSViv( (unsigned char)foo1 & 0x0f ));
              Inline_Stack_Done;
      }
      
      
      MODULE = index_pl_8d47  PACKAGE = main
      
      PROTOTYPES: DISABLE
      
      
      void
      woot (foo)
              char *  foo
              PREINIT:
              I32* temp;
              PPCODE:
              temp = PL_markstack_ptr++;
              woot(foo);
              if (PL_markstack_ptr != temp) {
                /* truly void, because dXSARGS not invoked */
                PL_markstack_ptr = temp;
                XSRETURN_EMPTY; /* return empty stack */
              }
              /* must have used dXSARGS; list context implied */
              return; /* assume stack size is correct */
      
      

      the generated .c file

      #index_pl_8d47.c
      /*
       * This file was generated automatically by xsubpp version 1.9508 from the 
       * contents of index_pl_8d47.xs. Do not edit this file, edit index_pl_8d47.xs instead.
       *
       *      ANY CHANGES MADE HERE WILL BE LOST! 
       *
       */
      
      #line 1 "index_pl_8d47.xs"
      #include "EXTERN.h"
      #include "perl.h"
      #include "XSUB.h"
      #include "INLINE.h"
      
      void woot(char *foo) {
              Inline_Stack_Vars;
              Inline_Stack_Reset;
              Inline_Stack_Push(newSViv( (unsigned char)foo[0] >>   4 ));
              Inline_Stack_Push(newSViv( (unsigned char)foo[0] & 0x0f ));
              Inline_Stack_Push(newSViv( (unsigned char)foo1 >>   4 ));
              Inline_Stack_Push(newSViv( (unsigned char)foo1 & 0x0f ));
              Inline_Stack_Done;
      }
      
      
      #line 27 "index_pl_8d47.c"
      XS(XS_main_woot)
      {
          dXSARGS;
          if (items != 1)
              Perl_croak(aTHX_ "Usage: main::woot(foo)");
          SP -= items;
          {
              char *  foo = (char *)SvPV(ST(0),PL_na);
      #line 26 "index_pl_8d47.xs"
              I32* temp;
      #line 38 "index_pl_8d47.c"
      #line 28 "index_pl_8d47.xs"
              temp = PL_markstack_ptr++;
              woot(foo);
              if (PL_markstack_ptr != temp) {
                /* truly void, because dXSARGS not invoked */
                PL_markstack_ptr = temp;
                XSRETURN_EMPTY; /* return empty stack */
              }
              /* must have used dXSARGS; list context implied */
              return; /* assume stack size is correct */
      #line 49 "index_pl_8d47.c"
              PUTBACK;
              return;
          }
      }
      
      #ifdef __cplusplus
      extern "C"
      #endif
      XS(boot_index_pl_8d47)
      {
          dXSARGS;
          char* file = __FILE__;
      
          XS_VERSION_BOOTCHECK ;
      
              newXS("main::woot", XS_main_woot, file);
          XSRETURN_YES;
      }
      
      

        Outstanding. Thanks. I thought that putting the code in a sub would hit the performance of the vec4 version.

        From what (little) I understand of what just scanned, I have to agree with your statement that (at this level at least) Inline C == XS.

        When I (eventually) succeed in building my own copy of perl, it will definately be "Inline C for me", for this sort of thing.

        Now if only I could find someone who could explain what this means....

        Info: resolving __sys_nerr by linking to __imp___sys_nerr (auto-import +) fu000001.o(.idata$3+0xc): undefined reference to `libmsvcrt_a_iname' nmth000000.o(.idata$4+0x0): undefined reference to `_nm___sys_nerr' dmake.exe: Error code 1, while making '..\miniperl.exe'

        Or rather, I know what an Unresolved Symbol is, but how to track down the source. I grepped and grepped for those weird filenames and can't find hide nor haor of them:(


        Examine what is said, not who speaks.

        The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

Re: Nybbles take to long
by zengargoyle (Deacon) on Feb 02, 2003 at 10:13 UTC

    built for AS 633 or 802!!!! sorry...

    use Inline C => <<'_C_'; void woot(char *foo) { Inline_Stack_Vars; Inline_Stack_Reset; Inline_Stack_Push(newSViv( (unsigned char)foo[0] >> 4 )); Inline_Stack_Push(newSViv( (unsigned char)foo[0] & 0x0f )); Inline_Stack_Push(newSViv( (unsigned char)foo[1] >> 4 )); Inline_Stack_Push(newSViv( (unsigned char)foo[1] & 0x0f )); Inline_Stack_Done; } _C_ print join(' ',woot($n)),$/; # Gives 15 0 0 15
    map_hex_split_unpack            5024/s                   --                    -26%                 -31%                           -32%                      -39% -75%
    map_ord_and_shift_split         6827/s                  36%                      --                  -7%                            -8%                      -17% -66%
    map_vec_lists_splice            7314/s                  46%                      7%                   --                            -1%                      -11% -64%
    map_unpack_shift_mult_and_list  7385/s                  47%                      8%                   1%                             --                      -10% -63%
    unpack_map_shift_and_list       8188/s                  63%                     20%                  12%                            11%                        -- -59%
    woot                           20210/s                 302%                    196%                 176%                           174%                      147%   --