From the ActivePerl User Guide:

"Uninstalling ActivePerl on Windows MSI Package

If you installed ActivePerl using the MSI installer, uninstall using the Windows Add / Remove Programs applet.

AS Package

If you installed ActivePerl using the AS installer, uninstall by deleting the directory in which ActivePerl was installed."

You may also want to remove the file association and registry settings as tye suggested.