that certain characters like & (there may be others) that are between escaped double-quotes be escaped with a ^ (carat).

Yes ... that was the bit of information I was missing.
I had tried using the stick on the on the > and the &, to no avail:

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = \"\>\&STDOUT\"; Dump
+ $arg;"
The system cannot find the path specified.
The system cannot find the path specified.
[download]

But using the carrot works fine:

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = \"^>^&STDOUT\"; Dump
+ $arg;"
SV = PV(0x2dcd2c) at 0x1c90564
  REFCNT = 1
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x1c8ff64 ">&STDOUT"\0
  CUR = 8
  LEN = 10
  COW_REFCNT = 1
[download]

Perhaps when the shell saw a backslash in those contexts, it might have been inclined to think "directory separator", so they had to come up with a different escape symbol.

UPDATE: It seems that, even with q()and qq(), there are still times when the carat needs to be invoked as an escape:

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = q(">&STDOUT"); Dump 
+$arg;"
>& was unexpected at this time.

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = q("^>^&STDOUT"); Dum
+p $arg;"
SV = PV(0x6bcd2c) at 0x57b30c
  REFCNT = 1
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x58056c ">&STDOUT"\0
  CUR = 8
  LEN = 10
  COW_REFCNT = 1

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = qq(">&STDOUT"); Dump
+ $arg;"
>& was unexpected at this time.

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = qq("^>^&STDOUT"); Du
+mp $arg;"

SV = PV(0x6ccd2c) at 0x1e30fe4
  REFCNT = 1
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x1e6f47c ">&STDOUT"\0
  CUR = 8
  LEN = 10
  COW_REFCNT = 1

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = q(\">&STDOUT\"); Dum
+p $arg;"
>& was unexpected at this time.

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = q(\"^>^&STDOUT\"); D
+ump $arg;"

SV = PV(0x55cd2c) at 0x650274
  REFCNT = 1
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x690f5c "\">&STDOUT\""\0
  CUR = 10
  LEN = 12
  COW_REFCNT = 1

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = qq(\">&STDOUT\"); Du
+mp $arg;"

>& was unexpected at this time.

C:\_32>perl -Mstrict -MDevel::Peek -le "my $arg = qq(\"^>^&STDOUT\"); 
+Dump $arg;
"
SV = PV(0x34cd34) at 0x1d8afbc
  REFCNT = 1
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x1d905ac "\">&STDOUT\""\0
  CUR = 10
  LEN = 12
  COW_REFCNT = 1

C:\_32>
[download]

Cheers,
Rob