Write a VB app and you'll have to install both the VB runtime and any ActiveX controls you need on target machines.

Now it is so, sadly. Thanks to OLE and ActiveX. But with VB3, one of the latest version of VB for 16-bits, it didn't need it: just put your executable, the VB runtime engine, and any VBX files you used (plus DLL's as used by the VBX'es behind the scenes) all in one directory on a CD-ROM, and you could run it on any PC which had a CD-ROM drive, without installation. It was a major advantage.

And with a perl application "compiled" into one executable with perl2exe (or likely PerlApp, though I have not tried that), you can now (still) do the same with a Perl script. Even if it includes XS modules, as compiled DLL files are included as well.