EXTERN.h is part of the perl C API which is needed for embedding the interpreter and building XS modules. The official perl distribution contains all these files, but many linux distributions don't bundle it as part of the base perl package (some even leave out a lot of core modules). There should be one or more additional packages for your distribution that supply the C API files. They're usually called "perl-*dev*" or something like it. (From googling, it looks like the mandrage package is "perl-devel-5.*.rpm")

update after seeing your update: installing the header files from another perl installation will only work reliably if both installs are from the exact same perl version. IFAIK there should be no real problems in copying over the header files from a windows install provided the versions are the same, but take note that the EXTERN.h file isn't the only file you'd need in general.

Also note that usually, the perl-dev packages don't contain a seperate perl runtime (interpreter etc.); they're just the files and programs you need to develop/compile programs/libraries against the perl binary - they're complimentary to the "base" install.

If you're writing your own C extensions (not just compiling third-party modules), I'd recommend compiling your own perl interpreter from source with all the debug info intact (I install my development perl in /usr/local, so I still have the systems standard perl install in /usr)