From what I read only using an asymmetric cipher would be very slow.

The question is whether you need to the speed enough to warrant that extra code. The extra code increases development time, the probability of a bug, the probability of a security bug, debugging time and maintenance time.

From the customer's perspective: No. The tool will only be used on rare occasion, and the few seconds lost on those occasions won't matter.

From your perspective: Maybe. Will you be receiving data from many customers at the same time? often? From what you said, it doesn't like it.

Since you have to do it anyway, start with trying just the asymetric portion. If your needs aren't satisfied, then add the symetric bit.