You might be interested in checking out http://win32.perl.org/, especially the Distributions, the Win32 Modules, and the Win32-friendly modules.

Personally, I've tried ActiveState, Cygwin perl, XAMPP+perl, and some other free no-install Apache+MySQL+SQLite+PHP+Perl for Windows that will run from CD (for which I cannot remember nor find a name, but its Perl is based on AS anyway).

Of all of these, I have a copy of ActiveState lying around, and I'm using a modified version of the no-install one I can't remember the name of for a couple of CD-based projects. I use Cygwin perl when I'm actually in Cygwin, which turns out not to be often because I have a Linux box and a Windows box both at the same desk each with their own keyboard and monitor. Cywin perl theoretically has the advantage that while within Cygwin, CPAN might work better and perl itself might behave closer to expectations since Cygwin is a saner approximation of POSIX than bare Windows. Mostly I use Strawberry Perl for the last several months.

Despite Strawberry Perl being called alpha, I've never had any stability problems. Most modules install via CPAN just fine, including many with XS binary portions. It's possible to get the command-line PPM working with it. I have done so, but I remember that was a bit of a pain. PPM can be important for a few modules that just don't install right with CPAN.pm yet. Most of the modules with which you might have problems are covered by the Vanilla Perl Problem Modules (Strawberry Perl is a superset, basically, of Vanilla Perl), and many have workarounds, patches, or at least more information about the issues. Most of the Win32 distributions have such a section of modules, some larger and some smaller. YMMV.

Update: fixed a typo