XML::Simple isn't made for complicated parsing. It's intended for simple configuration files, and has a couple of tricks thrown in to help in this regard.
I've used XML::Parser on my XML projects. I understand from subsequent reading that most consider that doing it the hard way, but it works for me.