LittleGreyCat has asked for the wisdom of the Perl Monks concerning the following question:

Gracious Monks,

if one had to handle data that was bit significant rather than byte significant, (and couldn't do it in 'C') what would be the best Perl tools to use?

The potential application is decoding and encoding bit significant data to/from a low bandwidth device which can't afford the luxury of the spare bits that come with byte orientated operations.

Many thanks

Dave R

Replies are listed 'Best First'.
Re: Handling bits in Perl
by ikegami (Patriarch) on Jul 25, 2006 at 14:41 UTC
    The same bit operations available to C (&, |, ^, << and >>) are available to Perl. Perl also has vec. If you plan on doing some serious number crunching, you could use PDL or interface to C code via XS or Inline::C.
[reply]
[d/l]
[select]
      If ikegami's excellent suggestions aren't enough, there's also a Bit::Vector module on CPAN.

      One nice thing about the bitwise operators in Perl is that you can use them on whole strings, not just on integers. For example, 

      use strict;
my $x = "abcdefghijklmnopqrstuvwxyz";
my $y = ("A" ^ "a") x length($x);
my $z = $x & ~$y;
print "\$z = $z\n";

      [download]
      This outputs $z = ABCDEFGHIJKLMNOPQRSTUVWXYZ. (This is just an example, not recommended for case conversion because it screws up non-alphabetic characters.)

      You should also be aware of pack/unpack "H*", pack/unpack "B*", or "b*, to print bit strings for debugging, to input long constants, etc.

[reply]
[d/l]
[select]
        Unforunately, you can't use << and >> on strings :(
[reply]
[d/l]
[select]
Re: Handling bits in Perl
by rodion (Chaplain) on Jul 25, 2006 at 16:38 UTC
    You say you have a data stream coming from a device. This which may mean there is some structure to the data stream. unpack() is a great tool and will go down to the single bit level with "B" and "b" specs. It can be used along with the tools already mentioned to do pretty much anything you want.

    Remember that you can use successive unpack()s to pick things apart and leave some, as in 

    use strict;

my $str = '567';
my ($ch, $rest);

($ch, $rest) = unpack('a a*', $str);
while (length $ch) {
   my $high_nib= ord($ch)>>4;
   my $low_nib = ord($ch) & 15;
   my ($high_bit,$case,$alpha,$rangel,$bit3,$bit2,$bit1,$bit0)
      = split '', unpack('B*',$ch);
   my @bits 
      = split '', unpack('B*',$ch);

   print "$ch == @bits   ...   ";
   printf "%d, %d, %d, %d : %d/%d\n",
      $high_bit,$case,$alpha,$rangel,$high_nib,$low_nib;   
   ($ch, $rest) = unpack('a a*', $rest);
}
     
# prints
# 5 == 0 0 1 1 0 1 0 1   ...   0, 0, 1, 1 : 3/5
# 6 == 0 0 1 1 0 1 1 0   ...   0, 0, 1, 1 : 3/6
# 7 == 0 0 1 1 0 1 1 1   ...   0, 0, 1, 1 : 3/7

    [download]
[reply]
[d/l]
      Most Gracious Monks,

      I read a bitstream from a Socket, running as a socket server. Data seems to arrive in streams of several kilobits, before the remote client closes the socket. (My study of http://www.perlmonks.com/index.pl?node_id=21054 suggests I should replace <> with sysread, in the hope of a more real-time read, but that is on the different topic of buffering and blocking.)

      I am using Bit::Vector on these long bitstreams. But on re-reading the manual, I see some Bit::Vector operations are limited to 32 bits. (Block_Read, for example.) I'd appreciate some words of enlightenment on the safe way to handle very long bitstreams.

      Should I laboriously process the bitstream 8-bits at a time, perhaps. Maybe using whatever is Bit::Vector's equivalent of rodion's previous posting on unpack( 'a a*', $rest )?

      Many fields in the definiton of the bitstream are more than 8 bits long, some over 16. The spectre on endian-ness is haunting me.

      Thank-you.

[reply]
[d/l]

Back to Seekers of Perl Wisdom