I've been looking through the ExtUtils::Manifest package and it does not appear that you can.
Line 189 (in the maniread sub) splits a MANIFEST line with this code:

my($file, $comment) = /^(\S+)\s*(.*)/;
[download]

It looks like anything after the first set of spaces is considered a comment.

Update:
After further investigation it looks like the open statement in the cp_if_diff sub might be the problem. The open statement is:

open(F,"< $from\0") or die "Can't read $from: $!\n";
[download]

If there was some way to represent a file with spaces using escape sequences (eg \x20, \040 ...) that would make this command open a file then it should work.