grep returns all elements for which the code block returns true. Like most languages, in Perl zero is considered false and one (non-zero) is true.
{$t ^= 1} is carrying out an XOR operation, which produces a 1 if either of its inputs is 1 and a 0 if both 0 or 1. Therefore 1 (true) is produced for the second (b) and forth (d) iterations.

Using just the number 1 means the result is true for all elements, so all are printed, while using zero (false) mean none are. Using $t directly is just another way of saying true, since its value is 1.