This is true only so long as the third party already has access to your account on the local machine. If they do, chances are a keylogger wouldn't be too hard a thing to use in case you weren't using public key encryption to authenticate to another box. That way, they still have access, and you are none the wiser. Besides, ssh logs pretty thoroughly.
If you don't trust public key encryption, remember never to trust SSL or TLS on a web site. Secure shell is no different.