use happens at compile time, so before the if gets checked.

Yes, I know you've put it in a BEGIN { ... } block, so the if also happens at compile time, but the use is happening at a compile-time within compile-time, if you see what I mean.

One solution is to not use use and use require;import instead.

BEGIN {
  if ($ENV{USE_XYZ} == 1) {
    require XYZ;
    XYZ->import();
  }
}
[download]

Another option is to take advantage of the core module if (if.pm).

use if $ENV{USE_XYZ}, "XYZ";
[download]

In rare cases, if you're loading something that does weird funky stuff at compile time, the first option won't work well. The second option should generally be okay.