If your doing this from C, then you might consider taking both Perl and the C-runtime out of the equation completely and going directly to the OS for your memory.

The OS has an API with which you can reserve a large amount of (virtual) memory without actually allocating it. You then make a second call to actually commit chunks of the reserved allocation as you need it. In this way, you can reserve a contiguous address space large enough to accomodate your biggest likely requirements and then grow the actual allocation into that reserved contiguous address space as you need to. This removes the need for any reallocation and the copying and (temporary) duplication that that entails and bypasses several intermediate levels of freespace chain chasing, coalesing and shuffling to boot.

As your search code is written in C, there is no great problem with utilising the address space provided by the OS for this purpose, you don't need for the overall space to be managed by Perl. You can simple copy your individual array elements to a Perl scalar for returning to the perl code once you have located it.

Needless to say I haven't done this from perl, but the C part is relatively trivial. I located this brief description and sample code on MSDN that demonstrates using this technique if the idea is of any interest.

---

Examine what is said, not who speaks.

"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
If I understand your problem, I can solve it! Of course, the same can be said for you.