For better or worse, Perl 5 is not going to be losing its refcounted implementation any time soon. The now-defunct Ponie project was an attempt to replace perl's memory management with Parrot's PMCs. While I have no doubts about the skills and competence of the programmers who attempted this heroic feat, it didn't work out as planned.

You can't rip out the guts of a mature code base and replace it with something else and expect everything to work. For instance, the internal layouts of SVs, AVs, HVs and the like have all recently received extensive changes to reduce their memory footprints (while keeping the public interface unchanged).

This work was completed months ago, and yet CPAN modules continue to surface that no longer work with blead, since they apparently have very definite ideas about what the bodies of these things should look like. And all things considered this is a minor issue compared to replacing the memory management infrastructure.

Still, if you are serious about this idea, you should think about an approach that allows this allocator to co-exist with the current system, for instance, asking for a large slab of memory from the underlying system and letting Hoard play with that (assuming you can prevent it from being unexpectedly realloc'ed).

You could then build a custom allocation framework for new code that knows about it and wants to use it. If you want to create this, go ahead. If you get stuck, ask on perl5-porters, and people will help you. I would hate you to have the impression that just because you think it's a good idea that someone else will write the code.

If you want to create this, go ahead.

Actually, as linked in my OP, I'm already developing a replacement for threads::shared, and hope to test it with Hoard.

I would hate you to have the impression that ... someone else will write the code.

If I gave the impression that I expected someone else to implement something, I apologize. My purpose is to elicit comments from anyone w/ Hoard experience, and possibly engender discussion of the possibilities. And yes, I have reviewed the malloc.c and threads::shared code.

From the tone of your post, I presume you have no experience with Hoard, nor any interest in seeing it applied to Perl ? Yet you still felt the need to post a condescending comment. I had hoped this was a welcome venue for this discussion; I hope Perlmonks isn't falling into the sort of elitism I've witnessed in other forums.

Perhaps it wasn't the response you wanted, but on the other hand I don't think grinder's post is negative; merely informational and cautionary.
Part of having a welcoming venue for discussion includes receiving comments and perspectives other than our own and I think grinder's comment meets that criterion.

[Jon]

You can't rip out the guts of a mature code base and replace it with something else and expect everything to work. For instance, the internal layouts of SVs, AVs, HVs and the like have all recently received extensive changes to reduce their memory footprints (while keeping the public interface unchanged).

replacing memory allocator isn't related to layout of SVs, yet Perl already have different binary incompatile ways of using different memory approaches, so using yet another one is straightforward,
see my other post on the thread

Without claiming to fully understand the issues involved, I have my doubts as to whether improving the performance of the underlying memory management, vis-a-vis things like cache-line sharing and fragmentation, would seriously have much affect upon the performance of iThreads.

I think the limitations on performance of iThreads centre on the duplication and proxying of shared entities and the belt & braces locking required by Perl datastructures. It's hard to see these very low level concerns having a dramatic affect on the overall performance.

It would be nice to be proved wrong though :)

For what it's worth, throw in the relative heavyweight Perl interpreter (and the work required to clone one), and I completely agree with your assessment.

Once I successfully built Perl together with Tcl/Tk both using Perl's memory allocator and found this gave me considerable speed improvement.

This experiment gave me an idea that using other memory manager is an easy task. You only need to override 3 or 4 functions and go figure out the results.

But Perl's memory allocator is *sooo* efficient that it hardly can be ever bitten, because it was written far in the past, when people bothered somehow on efficiency.

I have an experience that perl's own memory allocator is pretty good.

I agree...except that, in its zeal to provide optimal performance, it makes some choices wrt threads that create add'l issues for sharing memory between threads, namely, the thread that malloc()'s something must be the thread to free() it (in order to avoid the overhead of heap locking). Presumably, Hoard avoids this (just how, I haven't yet fully researched).

Which led to my curiousity if anyone had used Hoard (with Perl or anything else), and to ponder if it might be a means to achieve the performance wo/ the threading restrictions.

you're right WRT thread safety... but Perl's thread robustness will not increase with switchnig to threads-friendly memory allocator,
because, AFAIU switching off Perl memory allocator (so returning to default configuration) will use OS memory manager, which is thread safe by definition.

I have an experience that perl's own memory allocator is pretty good. I am curious why it is not *on* by default.

Apart from thread safety, there is the issue of "bug compatibility" with the system malloc when using buggy third party libraries. I've seen cases where third party libraries erroneously call free on a pointer that was not got from malloc, yet the system malloc is forgiving and does not crash. Now if you replace the system malloc with a pickier one that does crash ... In one way, that is good because you've just flushed out a bug in the third party library. OTOH, you may not have the source code for the third party library to fix it and the vendor may be unresponsive. Most third party libraries are tested primarily with the system malloc and work more reliably with it.

See also Using Perl's Malloc.