The parser XML::Simple underneath the hood uses makes a difference also. If you leave it at the default, it will use XML::SAX. XML::SAX by default will use XML::SAX::PurePerl. As implied by the name, it is an XML parser written in Perl.

For faster results, try installing XML::LibXML. You will need to install the library that it uses underneath the hood (libxml2) and it can be found here.