error due to "Can't locate xxxx.pm in @INC"

Ah, that makes more sense... You'd have to modify @INC from within your Perl code. Normally, to make Perl find modules in custom locations, you'd put

BEGIN {
    unshift @INC, ...list of directories where the modules are install
+ed... ;
}
[download]

at the top of your script. Or set the environment variable PERL5LIB appropriately. Or "use lib ...;" — though the latter will only work as soon as Perl can find the core module lib.pm (in other words, you'd have to employ one of the other techniques to point Perl to - at least - where lib.pm lives).

I don't know how your embedded interpreter is meant to operate (e.g. run one larger script, or many small code snippets/commands), but it might be easiest in your case to use PERL5LIB...

Note that you'll probably also need to ship some of the core modules with your application (core modules are the ones that come with a normal Perl installation). Which ones exactly will depend on what your code does. In case of doubt, just copy them over one at a time until you no longer get any "Can't locate ... in @INC" errors — maintaining the relative directory structure, of course, i.e. if you'd want to use Some::Module::Foo, the module would have to be found as Some\Module\Foo.pm relative to one of the directories you specified (with PERL5LIB, etc.).

Also, don't forget to copy over the respective DLLs that any XS modules (extension modules written in C) you use will need. As to where they need to go, just check how this is set up in a normal Perl installation, and replicate the same relative directory structure...

Lastly, I hope the installation path of your application is known at build time, otherwise you'd have to implement some code that determines the location of the binary, and sets up PERL5LIB/@INC accordingly...   (Perl binaries on Windows (at least the ActiveState build) typically have some code that does just that (i.e. set @INC dynamically), so you might want to investigate how they do it...   Side note: the reason that this mechanism doesn't seem to work in your case is probably due to the fact that the respective code isn't in perl510.dll, but rather in the wrapper (the actual perl binary) that loads perl510.dll — though this a just a guess... at the moment I don't have a Windows machine here to try.)