I'm back having done a lot of experimental work. I have more questions that would like to be answered. First, some code:

EXTERN_C void xs_init (pTHX);

using namespace std;

CPerlEngine::CPerlEngine(char* pScriptFile) :
        mInterpreter(NULL),
        mScriptFile(pScriptFile)
{
  this->mInterpreter = ::perl_alloc();
  assert (this->mInterpreter != NULL);

  PERL_SET_CONTEXT(this->mInterpreter);

  ::perl_construct(this->mInterpreter);

  char* theArguments[] = {"-x", "-S", "-s", pScriptFile};

  ::perl_parse(this->mInterpreter, 
        &xs_init, 
        4, 
        theArguments, 
        NULL);
  PL_exit_flags |= PERL_EXIT_DESTRUCT_END;

  ::perl_run(this->mInterpreter);
}

CPerlEngine::~CPerlEngine(void)
{
  ::perl_destruct(this->mInterpreter);
  ::perl_free(this->mInterpreter);
}


void
CPerlEngine::invoke(const char* pFunctionName,
    vector<string> pParameters) 
{
  assert (NULL != this->mInterpreter);

  PERL_SET_CONTEXT(this->mInterpreter);

  //  Pick up all the stack info in *this* threads local storage
  dTHX;

#ifdef PERL_CLONE_WORKS
  PerlInterpreter* newInterpreter = ::perl_clone(this->mInterpreter, 
  CLONEf_COPY_STACKS | CLONEf_KEEP_PTR_TABLE | CLONEf_CLONE_HOST);
//  PerlInterpreter* newInterpreter = ::perl_clone(this->mInterpreter,
+ CLONEf_CLONE_HOST);
//  PerlInterpreter* newInterpreter = ::perl_clone(this->mInterpreter,
+ NULL);
#else
  PerlInterpreter* newInterpreter = this->mInterpreter;
#endif

  assert (NULL != newInterpreter);

  ::perl_run(newInterpreter);

  dSP;
  ENTER;
  SAVETMPS;
  PUSHMARK(SP);

  for (vector<string>::iterator theIterator = pParameters.begin(); 
    theIterator != pParameters.end(); 
    theIterator++)
  {
    if (theIterator->length() > 0)
    {
      XPUSHs(::newSVpv(theIterator->c_str(), theIterator->length()));
    }
  }

  PUTBACK;
  ::call_pv(pFunctionName, G_DISCARD);
  FREETMPS;
  LEAVE;

#ifdef PERL_CLONE_WORKS
  ::perl_free(newInterpreter);
#endif
}
[download]

This is my C++ class wrapping the Perl interpreter(s). It's pretty simple. You instantiate it, it loads the specified script, and runs all of the global bits (please excuse any incorrect terminology). So far so good. The global bits are essentially a whole heap of "use blah" type statements. I think there's value in this because it results in a Perl interpreter with script loaded and references loaded - ready to be cloned and executed at will.

The idea, then, is to call invoke(...) passing the name of the Perl sub to call and some arbitrary number of string elements. This is all pretty cool and seems to work in multiple threads with a basic Perl script.

If I don't define PERL_CLONE_WORKS, the whole thing works like a bought one (actually better than many) but only in a single thread (obviously).

Now imagine I define PERL_CLONE_WORKS, instantiate CPerlEngine in one thread, and call invoke(...) on a separate thread. The new thread comes along, clones the existing interpreter, and then calls my sub. This all works perfectly with a basic script BUT with a more complex script (my cut-down spamd) I get a runtime crash out of the Perl Engine in VMem::Free(void* pMem) where it says

Perl_warn(aTHX_ "Free to wrong pool %p not %p",this,ptr->owner);

Note that this is not the global cleanup error and it's happening a long time before the end of the script and a reasonable distance into the script.

I think you (BrowserUk) have seen this before in different circumstances.

My fallback plan is to create a whole new PerlInterpreter using ::perl_alloc and load absolutely everything from scratch each or have my pool of interpreters (as previously discussed).

The value in being able to clone on the fly like this is that it won't need to have 50 interpreters lying around consuming vast amounts of memory waiting for something to come in.

Now to my questions:

1. Is my class doing everything it should do? If not, what else should it be doing?
2. I've tried all combination of flags on ::perl_clone and none work. For future reference, what flags should I be using?
3. What is

   Free to wrong pool

   telling me?

We're getting closer at least. Another couple of weeks of this sort of questioning and I might get there <g>.

Phil