Turns out it was my GREP_OPTIONS environment variable. I had this in my .bashrc

export GREP_OPTIONS="--exclude-from ~/.grep_excludes --color"

Seems some files were accidentally matching my excludes rules and the color was throwing off something else. Removing that fixed everything up (except for a single failure in t/porting/exec-bit.t).