The problem is when macro ... is passed a non-literal string

Good catch.
I had seen the comments in handy.h, and noted the absence of quotes in perlboy_emeritus' build - but still failed to put it all together properly.
(I had myself thinking that the strange ("" s "") construct must have been magically enforcing the requisite stringification but, of course, it merely throws the error if s is not a string.)

So the bug is in the version of Module::Build that perlboy_emeritus has installed - and any perl extension that is built with ExtUtils::MakeMaker (instead of Module::Build) will be free from this issue.
The version of Module::Build that I was using (on Windows) is 0.4214, which was fine - so I guess another alternative might have been to simply update Module::Build to a later version.

Here's a little C demo that I ran just to satisfy my limited understanding:

C:\_32\C>type try.c

#include <stdio.h>

int main (void) {
#define FOO(s) printf("%s\n", "" s "");
 FOO(VER);

 return 0;
}

C:\_32\C>gcc -o try.exe -DVER="\"0.1\"" try.c

C:\_32\C>try
0.1

C:\_32\C>gcc -o try.exe -DVER=0.1 try.c
try.c: In function 'main':
try.c:5:2: error: expected ')' before numeric constant
[download]

Cheers,
Rob