Some other annoying network quirks that I've run into:

• Ethernet collisions (okay, this one's easy... look for a segment in your trace that's unswitched, and see if the hub's blinking too much)
• Network card on fire (look for the smoke)
• Incorrect autonegotiation (one thinks it's full duplex, the other thinks half duplex)
• Too many hops on an ethernet segment. (should not go through more than 3 repeaters... shouldn't be a problem on a switched network).
• Poorly seated cables. (pull them all out, push them back in fully).
• Crosstalk (or other inductance). (not typically an issue, unless you're running incorrectly spec'd cables, or near phone/power lines)

More often than not, though, it's either a saturated network, or like etcshadow said, and it's a bad cable (poorly made, crushed, improper radius, nicked, etc.) or misconfigured. (bad hardware happens, but once you're burned in, it's fairly rare in my experience)

If you can sniff the network on either end, I'd suggest doing it, and see if you can spot anything wrong. You also have to remember that you're not running a real time OS (very few people are, as they can cause worse overall performance in most situations), so you can't be assured that anything is going to happen within a certain time frame.