I've been writing a lot of software lately that deals with direct hardware access (specifically analog and digital hardware for the Raspberry Pi). This means that I've had to learn some C, as well as get proficient with bit manipulation and the bitwise operators.

As part of my learning, I thought I'd write a module to do this bit manipulation for me, hence Bit::Manip was born. (There's also a Bit::Manip::PP for those who can't/don't want to use XS. It should be indexed shortly).

Here's a scenario based example of how the software can be used.

You have a 16-bit configuration register for a piece of hardware that you want to configure and send in. Here's the bit configuration:

|<--------- 16-bit config register ---------->| | | | |---------------------------------------------| | | | | | | |<------Byte 1: Control------>|<-Byte0: Data->| | | | |-----------------------------|---------------| | 15 | 14 13 | 12 11 | 10 9 8 | 7 6 5 4 3 2 1 | __ _____ _____ ______ _____________ ^ ^ ^ ^ ^ | | | | | START | | UNUSED DATA CHANNEL | PIN SELECT

...and the bit configuration:

15: Start conversation 0 - do nothing 1 - start conversation 14-13: Channel selection 00 - channel 0 01 - channel 1 11 - both channels 12-11: Pin selection 00 - no pin 01 - pin 1 11 - pin 2 10-8: Unused (Don't care bits) 7-0: Data

Let's start out with a 16-bit word, and set the start bit. Normally, we'd pass in an actual value as the first param ($data), but we'll just set bit 15 on 0 to get our initial data.

my $data = bit_on(0, 15);

A couple of helper functions to verify that we indeed have a 16-bit integer, and that the correct bit was set:

say bit_count($data); say bit_bin($data);

Output to ensure we're good:

16 1000000000000000

Now, we've got the conversation start bit set in our register, and we want to set the channel. Let's use both channels. For this, we need to set multiple bits at once. The datasheet says that the channel is at bits 14-13. Take the LSB (13), pass it along with the data to bit_set(), the number of bits we're intending to update (2) and as the last parameter, put the binary bit string that coincides with the option you want (0b11) for both channels):

# setting channel $data = bit_set($data, 13, 2, 0b11); # result: 1110000000000000

We'll use pin 1, and per the datasheet, that's 0b01 starting from bit 11 (again, we're setting 2 bits):

# setting pin $data = bit_set($data, 11, 2, 0b01); # result: 1110100000000000

The next two bits are unused, so we'll ignore them, and set the data. Let's use 186 as the data value (10111010 in binary). We're setting 8 bits here:

# setting data $data = bit_set($data, 0, 8, 186); # or: bit_set($data, 0, 8, 0b10111010); # result: 1110100010111010

Now we realize that we made a mistake above. We don't want both channels after all, we want to use only channel 1 (value: 0b01). Since we know exactly which bit we need to disable (14), we can just turn it off:

$data = bit_off($data, 14); # result: 1010100010111010

(You could also use bit_set() to reset the entire channel register bits (14-13) like we did above).

Let's verify that we've got the register configured correctly before we send it to the hardware. We use bit_get() for this. The 2nd and 3rd parameters are MSB and LSB respectively, and in this case, we're only checking the 1-bit start bit, so MSB and LSB are the same:

my $value = bit_get($data, 15, 15); say bit_bin($value); # result: 1

So yep, our start bit is set. Let's verify the rest:

# data # (note no LSB param. We're reading from bit 7 through to 0). # since we readily know the data value in decimal (186), we don't # need to worry about the binary representation say bit_get($data, 7); # result 186 # channel say bit_bin(bit_get($data, 14, 13)); # result 1 # pin select say bit_bin(bit_get($data, 12, 11)); # result 1 # ensure the unused bits weren't set say bit_get($data, 10, 8);

So now we've set up all of our register bits, and confirmed it's ready to be sent to the hardware for processing.

Perl has some handy features for making binary conversions very easy:

# use a binary number directly: say 0b101; # 5 # format a number into a binary string, and print it printf("%b\n", 254); # 11111110 # convert a number to a binary string, and assign it my $bin = sprintf("%b", 127); # 1111111

Disclaimer: also posted at blogs.perl.org.

update: updated to reflect a bug fix talked about later in the replies./update

Replies are listed 'Best First'.
Re: Bit string manipulation made easy with Bit::Manip
by VinsWorldcom (Prior) on Jan 25, 2017 at 18:33 UTC

    I've been using Bit::Vector for similar bit groupings within a 16-bit value when doing TCP/IP header manipulation with Net::Frame, its sub-modules and Perl Packet Crafter.

    I always found Bit::Vector a bit hard to understand, but it did what I needed. Your approach seems a bit more flexible and friendly.

      Thanks!

      I honestly wrote this just for practice, not because I needed software to do the bit mangling for me (as I'm getting quite good at both implementing and reading it), so I honestly didn't even look whether there were other modules that do this sort of thing before I started.

      I just thought that it may help others (or even myself if this stuff falls out of my brain in the future) :)

      ps. There's a bug in bit_set() that I've had to fix in the next version, which requires it to accept an extra parameter, so any code written against 0.02 will break if the next release comes out).

        Mentioned bug is fixed in Bit::Manip v1.01 and Bit::Manip::PP v1.00. Essentially, I had to add an additional param to bit_set():

        ... my $lsb = 2; my $num_bits_to_change = 3; my $value = 0b001; $data = bit_set($data, $lsb, $num_bits_to_change, $value); ...

        What was happening is that the leading zeroes in $value (0b001)) were not being counted as the length of the number of bytes to modify. The code strips all leading zeroes, so the number of bits would turn out to be one, whereby we need to change three bits. The first two (MSBs) get updated to be 0, and the last (the LSB specified) goes to 1.

        _
Re: Bit string manipulation made easy with Bit::Manip
by pryrt (Abbot) on Jan 30, 2017 at 21:03 UTC

    While looking at the documentation to bit_set() to make sure my "not really necessary" was true, I had some thoughts:

    Instead of calling the argument for the number of bits "$bits", I'd be tempted to document it as "$nbits"; some might want to misinterpret that argument name as a bit-mask instead of a number of bits, despite the text on the next line. (ie, when I first saw that, my mind set the "need more information flag" until I got to the next line. (This will especially help clarify if you end up implementing the next suggestion.)

    With longer words, there have been times when I want to address multiple disjoint bits simultaneously, such as when a hardware designer separates bits that would logically be grouped. For example, suppose you had a register where in the cheap version of a part, you set a 4bit offset and an offset-enable, where the enable is in bit 4, and the 4 bit offset is bits[3:0] = offset[3:0]; but in a more expensive but pin-and-register compatible version, the same register has an additional 2 bits of offset above the sign bit, so it's bits[6:5] = offset[5:4], bits[4] = enable, bits[3:0] = offset[3:0]. (I design tests for hardware for a living; I've seen much more convoluted register words than that.) To that end, I would want

    • bit_mask_disjoint(@bitlist) - where @bitlist (or $bitlist_ref if you prefer) tells which bit numbers should be affected
    • bit_set_disjoint($data, $bitmask or \@bitlist, $value) - spreads out the value across the bits that are set in $bitmask, or to the bits indicated by the array reference \@bitlist: yes, this second adds complication... but it's useful for users who think of certain controls being in a different order than the designer does.
    • Thus, for my example above:
      my $data = 0x00; bit_on($data, 4); # enables the offset my $offsetBits = bit_mask_disjoint(6,5,3,2,1,0); foreach my $val ( 0..63 ) { bit_set_disjoint($data, $offsetBits, $val); # sets the offset + bits without affecting the enable bit ... } ... my $ctrl = 0x00; # ctrl is a three-bit register my $permuteBits = [1,0,2]; # but HW designer and I think of the bits +as being in a different order foreach my $val ( 0..7 ) { bit_set_disjoint($ctrl, $permuteBits, $val); # 0:000; 1:010; 2: +100; 3:110; 4:001; 5:011; 6:101; 7:111; ... }

    (I don't particularly like the _disjoint() naming convention... you might want to see if you can come up with a better one.)

      Thanks for the superb feedback based on the appreciated work you did reading through the docs and playing with the code!

      I'm just prototyping a new chip using this software almost exclusively for my bit-banging just to test how it works at the moment. I've found a few things that irk me that I'll be looking at changing, but after I'm done with the guts of this hardware and I have a bit schematic ready to write in C, I'll carefully review your node here and consider with what I've found and most likely do another update.

      As it stands, I'm getting some Testers failures on v5.6 anyhow (import wasn't exported from Exporter back then, so I think I need to declare a specific version of that dist, so a fix is in order anyhow. As an aside, I shouldn't have gone to v1.xx so quickly. Normally I give myself some time to ensure no API changes occur. It is what it is.

        First full IC prototype using Perl, with the Bit::Manip software for bit banging, worked out quite well. Now I'm just finishing up the C code that will take over the work, and publish the distribution for the MCP4922 (well, the MCP49x2) digital to analog converters.

        update: breadboard layout and schematic used for the prototype testing./update

        After that's complete, I'll be making some of the suggested updates to Bit::Manip. Again, thanks for the feedback everyone!

        use warnings; use strict; use feature 'say'; use Bit::Manip qw(:all); use RPi::WiringPi; use RPi::WiringPi::Constant qw(:all); my $spi_chan = 0; my $pi = RPi::WiringPi->new; my $adc = $pi->adc; my $spi = $pi->spi($spi_chan); my $cs_pin = $pi->pin(18); my $shdn_pin = $pi->pin(6); # light up some pins per the datasheet # device channel select (CS) to HIGH, in case the # device started it as LOW. In this case, we # bit-bang the CS pin, instead of using the # hardware SPI $cs_pin->mode(OUTPUT); $cs_pin->write(HIGH); # device 'shutdown' (SHDN) pin we'll tie to HIGH, # when tied to HIGH, means all DACs active $shdn_pin->mode(OUTPUT); $shdn_pin->write(HIGH); # show the current voltage output % on both DAC # channels before we begin say $adc->percent(0); say $adc->percent(1); # dac 0 say "\nDAC 0...\n"; dac_write(0, [0b1111, 0b11111111]); say "dacA: " . $adc->percent(0) . "%"; dac_write(0, [0b0111, 0b0]); say "dacA: " . $adc->percent(0) . "%"; dac_write(0, [0b0, 0b0]); say "dacA: " . $adc->percent(0) . "%"; # dac 1 say "\nDAC 1...\n"; dac_write(1, [0b1111, 0b11111111]); say "dacB: " . $adc->percent(1) . "%"; dac_write(1, [0b0111, 0b0]); say "dacB: " . $adc->percent(1) . "%"; dac_write(1, [0b0, 0b0]); say "dacB: " . $adc->percent(1) . "%"; sub dac_write { my ($dac, $data) = @_; die "\$dac param must be 0 or 1\n" if $dac != 0 && $dac != 1; # init the register my $register = [0, 0]; # DAC (bit 7) (a/b == 0/1) we're writing to $register->[0] = bit_set($register->[0], 7, 1, $dac); # BUFFERING (bit 6) == 0 $register->[0] = bit_set($register->[0], 6, 1, 0); # GAIN (bit 5) == 1 $register->[0] = bit_on($register->[0], 5); # SHDN (shutdown) (bit 4) == 1 $register->[0] = bit_set($register->[0], 4, 1, 1); # DATA (bits 3-0) $register->[0] = bit_set($register->[0], 0, 4, $data->[0]); say "byte1: ".bit_bin($register->[0]); # DATA byte 2 $register->[1] = bit_set($register->[1], 0, 8, $data->[1]); say "byte2: ".bit_bin($register->[1]); # drop chip select to LOW to start conversation with the DAC $cs_pin->write( LOW ); # write our bytes to the SPI bus $spi->rw($register, 2); # go HIGH to tell the IC we're done clocking in bits $cs_pin->write( HIGH ); } $pi->cleanup;

        output:

        0.00 0.00 DAC 0... byte1: 111111 byte2: 11111111 dacA: 99.94% byte1: 110111 byte2: 0 dacA: 43.76% byte1: 110000 byte2: 0 dacA: 0.00% DAC 1... byte1: 10111111 byte2: 11111111 dacB: 100.00% byte1: 10110111 byte2: 0 dacB: 43.70% byte1: 10110000 byte2: 0 dacB: 0.00%

      I've done a mock up of what I think you're after. The function that handles the out of order sequence is bit_set_seq() below, and the one that'll handle missing bits (and leave them as is) is bit_set_skip(). Let me know if the output is what you expected. The implementation details aren't really relevant right now, as I just wanted to ensure the output was correct before I made it better.

      use warnings; use strict; use feature 'say'; use Bit::Manip qw(:all); # out of order sequence for (0..11){ bit_set_seq(0x00, [3, 1, 2, 0], $_); } # skip over for (0..11){ my $b = $_; $b = bit_on($b, 1); $b = bit_on($b, 2); bit_set_skip($b, [0, 3], $_); } sub bit_set_seq { my ($b, $seq, $val) = @_; my $lsb = (sort @$seq)[0]; my $msb = (sort @$seq)[-1]; my $nbits = ($msb - $lsb); $b = bit_set($b, $lsb, $nbits, $val); say "** $val"; printf("seq before: %b\n", $b); my $x = $b; for (0..$nbits){ my $o = bit_get($b, $_, $_); my $s = bit_get($b, $seq->[$_], $seq->[$_]); if ($o != $s){ $x = bit_set($x, $_, 1, $s); } } printf("seq after %b\n\n", $x); } sub bit_set_skip { my ($b, $seq, $val) = @_; my $lsb = (sort @$seq)[0]; my $msb = (sort @$seq)[-1]; my @leave_bits = map $seq->[$_-1]+1..$seq->[$_]-1, 1..@$seq-1; my %leave_vals = map {$_ => bit_get($b, $_, $_)} @leave_bits; my $nbits = ($msb - $lsb) + 1; $b = bit_set($b, $lsb, $nbits, $val); say "** $val"; printf("skip before: %b\n", $b); for (keys %leave_vals){ $b = bit_set($b, $_, 1, $leave_vals{$_}); } printf("skip after: %b\n\n", $b); }

      output:

      ** 0 seq before: 0 seq after 0 ** 1 seq before: 1 seq after 1000 ** 2 seq before: 10 seq after 10 ** 3 seq before: 11 seq after 1010 ** 4 seq before: 100 seq after 100 ** 5 seq before: 101 seq after 1100 ** 6 seq before: 110 seq after 110 ** 7 seq before: 111 seq after 1110 ** 8 seq before: 1000 seq after 1 ** 9 seq before: 1001 seq after 1001 ** 10 seq before: 1010 seq after 11 ** 11 seq before: 1011 seq after 1011 ** 0 skip before: 0 skip after: 110 ** 1 skip before: 1 skip after: 111 ** 2 skip before: 10 skip after: 110 ** 3 skip before: 11 skip after: 111 ** 4 skip before: 100 skip after: 110 ** 5 skip before: 101 skip after: 111 ** 6 skip before: 110 skip after: 110 ** 7 skip before: 111 skip after: 111 ** 8 skip before: 1000 skip after: 1110 ** 9 skip before: 1001 skip after: 1111 ** 10 skip before: 1010 skip after: 1110 ** 11 skip before: 1011 skip after: 1111

        I'm not sure I correctly interpreted your intended API on bit_set_seq(), and I know I didn't on bit_set_skip.

        For bit_set_seq(), I interpreted the index to the array-ref as the bit number of the output, and the value at that index as the bit number for the input. If you intended the other way around, my tests below should work for exp0123 and exp3210, but won't be right for exp0312. But since my tests are completely failing, there's probably something I'm missing about the way you intended it to be used.

        use Bit::Manip qw(:all); use Test::More; # out of order sequence my @exp0123 = (0..15); my @exp3210 = reverse(0..15); my @exp0312 = (0b0000,0b0001,0b0100,0b0101,0b1000,0b1001,0b1100,0b1101 +,0b0010,0b0011,0b0110,0b0111,0b1010,0b1011,0b1110,0b1111); for (0 .. 15) { is( bit_set(0x00, 2, 4, $_), $_ << 2, "nomal set, shifted up 2" ); is( bit_set_seq(0x00, [0,1,2,3], $_), $exp0123[$_], "[0,1,2,3]: I +expect bit 0 [LSB] of input to be transferred to bit 0 [LSB] of outpu +t -- ie, same order"); is( bit_set_seq(0x00, [3,2,1,0], $_), $exp3210[$_], "[3,2,1,0]: I +expect bit 3 [MSB] of input to be transferred to bit 0 [LSB] of outpu +t, in2 -> out1, in1 -> out2, in0 -> out3 -- ie, reverse the bits"); is( bit_set_seq(0x00, [0,3,1,2], $_), $exp0312[$_], "[0,3,1,2]: I +expect in0 -> out0, in3->out1, in1->out2, in2->out3"); } ... # your subs go below, not quoted here...

        For bit_set_skip, I couldn't interpret it at all... What I would want would be a feature where I could put arbitrary gaps, for example, $out = bit_set_skip( $in, $seq, $val ). I would want to break $val into chunks based on seq, and put it into $in to create out. For this table: K means "keep the same bit value as in input", whereas a 1 or 0 means that I want that value there, regardless of what was originally in INPUT. I couldn't figure out the $seq necessary to do this, but below my table is the API I would have guessed to set this...

        # INPUT VAL DESIRED OUTPUT # 0bKKKK_KKKK 0b0000_0000 0bKKK0_K000 # 0bKKKK_KKKK 0b0000_0001 0bKKK0_K001 # 0bKKKK_KKKK 0b0000_0010 0bKKK0_K010 # 0bKKKK_KKKK 0b0000_0011 0bKKK0_K011 # 0bKKKK_KKKK 0b0000_0100 0bKKK0_K100 # 0bKKKK_KKKK 0b0000_0101 0bKKK0_K101 # 0bKKKK_KKKK 0b0000_0110 0bKKK0_K110 # 0bKKKK_KKKK 0b0000_0111 0bKKK0_K111 # 0bKKKK_KKKK 0b0000_1000 0bKKK1_K000 # 0bKKKK_KKKK 0b0000_1001 0bKKK1_K001 # 0bKKKK_KKKK 0b0000_1010 0bKKK1_K010 # 0bKKKK_KKKK 0b0000_1011 0bKKK1_K011 # 0bKKKK_KKKK 0b0000_1100 0bKKK1_K100 # 0bKKKK_KKKK 0b0000_1101 0bKKK1_K101 # 0bKKKK_KKKK 0b0000_1110 0bKKK1_K110 # 0bKKKK_KKKK 0b0000_1111 0bKKK1_K111 for (0 .. 15) { is( bit_set_skip( 0x00, [0,1,2,4], $_), (($_ & 0x08)<<1) | (($_ & +0x07)<<0), "put a gap between the MSB and the lower three bits"); }
Re: Bit string manipulation made easy with Bit::Manip
by jmlynesjr (Deacon) on Jan 30, 2017 at 18:17 UTC

    Steve:

    You have set, on, & off, should you also have clr?

    James

    There's never enough time to do it right, but always enough time to do it over...

      Hmm, … but you could just do $data = 0; or did I understand you wrong?

      Do you mean clr on a single bit? ie. do you mean 'unset' (or zero) it?, or do you mean the entire byte? I can definitely add that for completeness, but as soonix stated, $data = 0; will do it too.

        My thought was that James wanted _clr is to _set as _off is to _on... But that's not really necessary, since all you need to do is _set($data, $lsb, $bits, 0).