To uninstall, I would just manually delete the files that were installed by 'make install'. (These files would have been named during the 'make install' process.)
It's generally unnecessary to remove anything - you can just leave them sit there and they'll do no harm. If you want to make modifications, just modify the source, and then build and install again. (The new install will overwrite the old installation.)