Does it convert the decimal number 2 to binary and then shift each bit?

Yes. Consider:

my $num = 22     # 0b00010110
print $num << 2; #   0b01011000
[download]

$num << 2 has two zeros added to the right, and two removed from the left. The binary pattern has been shifted two bits to the left.

Update

You may find it helpful to play with the following

perl -wE 'printf "%#010b", 33 # 0b00100001'
[download]

(The documentation for printf's formatting is described in sprintf)