The other two answers are good (and way better than my approach), but I'd share my thoughts anyway.

As .xpm data is extremely easy to analyse, I'd do

$ convert -transparent-color Black file.png file.xpm
[download]

Where Black should be replaced with a color not in the image. Then analyze the .xpm and use that to crop the image.

To be completely dynamic, you can repeat the process for Black, White, and Red and take the minimum margins of the three analises.