in reply to Working with BIG binary fields

if you need a really fast way, you might create an array with 256 entries, in which every possible entry aka byte points to an array of the set bits in it:

my @SetBits= ( [], [0], [1], [0,1], [2], [0,2],
  [1,2], [3], [0,3] ...

...

sub findsetbits { #feed with array of bytes
  my @result;
  my $offset=0;
  foreach my $byte (@_) {
    foreach (@{$SetBits[$byte]}) {
      push @result, $_+$offset;
    }
    $offset+= 8;
  }
  return @result;
}

[download]

(Untested)

Replies are listed 'Best First'.
Re^2: Working with BIG binary fields
by AnomalousMonk (Archbishop) on Sep 21, 2010 at 00:28 UTC

    Here's an automated way to create the lookup table. Bit-counting function assumes data passed as raw binary string of any length, but test statements show a way to convert from hex-digit text string if it isn't. 

    >perl -wMstrict -le
"my @setbits;
 for my $byte (0 .. 255) {
   my $bits = unpack 'B*', chr $byte;
   my @offsets;
   push @offsets, $-[1] while $bits =~ m{(1)}xmsg;
   push @setbits, [ @offsets ];
   }
 sub find_set_bits {
   my ($raw_binary) = @_;
   my @offsets;
   my $offset = 0;
   for my $byte (unpack 'C*', $raw_binary) {
     push @offsets, map $_ + $offset, @{ $setbits[$byte] };
     $offset += 8;
     }
   return @offsets;
   }
 my $str = qq{ 01 02 03 \n 04 00 E0 0a 00 00 00 00 00 00 00 00 00 \n};
 (my $hexdigits = $str) =~ s{[^[:xdigit:]]}{}xmsg;
 my $raw = pack 'H*', $hexdigits;
 my @ones_offsets = find_set_bits($raw);
 print qq{'$str'};
 print qq{no 1-bits found} unless @ones_offsets;
 print qq{1-bit found at offset $_} for @ones_offsets;
"
' 01 02 03
 04 00 E0 0a 00 00 00 00 00 00 00 00 00
'
1-bit found at offset 7
1-bit found at offset 14
1-bit found at offset 22
1-bit found at offset 23
1-bit found at offset 29
1-bit found at offset 40
1-bit found at offset 41
1-bit found at offset 42
1-bit found at offset 52
1-bit found at offset 54

    [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom