Be sure that your xml source have the right encoding, that is ISO-8859-1 for latin1 ( including french characters ). I dont know if it's the solution, but maybe it's a clue =)
Unfortunately this doesn't seem to make any difference. I had a play with the different styles of XML::Parser and most of them seem to do the same, just deleting the entities. If I provide my own handlers the entities appear as raw character data.