It's possible you're being bitten by the HP-UX caveat of maxdsiz/maxdsiz64. This variable limits the maximum size any single process can grow to, and no amount of recompiling will fix it. Run "kmtune -q maxdsiz" to see whether the process limit is causing this. If so, then fiddling with kmtune or "Kernel Configuration" from within SAM will allow you to change it.