note the trailing non-URL characters in some of returned URLs

The problem is that those are valid URL characters, so Regexp::Common is doing the right thing, just not what you want it to do. URI::Find::Rule is applying a heuristic to take less than the longest possible URL, but a more natural stopping place.