This program probably was designed to do everything in-memory such that it fails when the files grow even slightly large. Unfortunately a redesign is in order. Throwing silicon at the problem might be a stopgap .. and, if so, might be the right thing to do. Chips are cheap.
Chips might be cheap (and the cost of labor substantially lower) but it has been my experience that such a stopgap rarely avoids the need to bite the bullet and do a proper design/redesign in the end.