Well, they say the devil is in the details.

It is certainly true that in summer days are longer at higher latitude and in winter they are shorter. If the continuum hypothesis held, there would definitely be a day (around but not on the equinox) when both were the same. But the continuum hypothesis does not hold, because day length between summer and winter is not a continuous function but a discrete one, so you can't guarantee a day when they are exactly the same, only a day when they are "pretty close."

TL;DR

There are three things that make the magic day different than the equinox:

1. As previously mentioned, the equinox occurs when the ecliptic longitude of the center of the Sun is at 0 or 180 degrees, but sunrise and sunset are the moments when the upper limb of the Sun is on the horizon. Since the apparent diameter of the Sun's disk is 0.5 degrees, this represents a difference of 0.25 degrees in position.
2. Atmospheric refraction causes distant objects to appear higher in the sky than they actually are. Obviously the magnitude of this effect depends on precise atmospheric conditions, but the accepted "typical" value is about 0.5 degrees near the horizon. This effect applies to Sunrise, but not the time of equinox.
3. Unless you are on the Equator, the Sun does not rise "straight up," but at an angle from the vertical equal to your latitude. This affects the amount of time it takes for the Sun to (apparently) travel the 0.75 degrees of arc from apparent Sunrise (when we see the upper limb of the Sun) and what I guess you would call geometric Sunrise (when the center of the Sun's disk is on the plane of the horizon).

I have not thought particularly about why the day changes length at different rates at Sunrise and Sunset, but I think it has to do with the fact that usually (meaning, not at equinox) the ecliptic (i.e. the path the Sun appears to take across the sky) intersects the horizon at a different angle at Sunrise than at Sunset (see point 3 above).

Disclaimer: I am strictly an amateur in the realm of computational astronomy. I believe the above to be basically correct, but I may have garbled the details. I am also the author of (among other things) Astro::Coord::ECI::Sun and friends, which (if you want to roll up your sleeves and get your hands dirty) can be used to investigate the above phenomena quantitatively. The POD says where the computations come from if you want to dig deeper.