Another way of doing it: the List::Util::reduce function works more or less the way you expected join to work, except that it's more general and not only for joining

use List::Util 'reduce';
print reduce { $a . ($b ? ":" : "") . $b } (0,1,0,1);
[download]

From perldoc List::Util :

 reduce BLOCK LIST
           Reduces LIST by calling BLOCK multiple times, setting $a and $b
           each time. The first call will be with $a and $b set to the first
           two elements of the list, subsequent calls will be done by setting
           $a to the result of the previous call and $b to the next element in
           the list.

           Returns the result of the last call to BLOCK. If LIST is empty then
           "undef" is returned. If LIST only contains one element then that
           element is returned and BLOCK is not executed.

               $foo = reduce { $a < $b ? $a : $b } 1..10       # min
               $foo = reduce { $a lt $b ? $a : $b } 'aa'..'zz' # minstr
               $foo = reduce { $a + $b } 1 .. 10               # sum
               $foo = reduce { $a . $b } @bar                  # concat

List::Util is a core module since perl-5.7.3.