I have built several applications in which the URLs that are generated for a page consist of a URL consisting simply of GUID=xxxx-xxx-xxxx.....   As the page links are generated within the application, they are stored in a hash by GUID – generated on-the-spot for one-time use – and that hash is saved in the session information.   This creates an impenetrable href, but also one that cannot be meaningfully “farmed,” either by Google or by the competition.   (The Google-unfriendliness can be viewed as either a good thing, or a bad thing, as the business case may be.)

A hybrid technique might use a more “Google friendly” URL structure, e.g. like the one commonly used by WordPress or Moveable Type based forums, but with the addition of such a code.   If the code is omitted or is not valid for any reason, the software might present a “sanitized” version of the page.   If you generate an XML sitemap file, it would omit all codes, so that Google would not generate any links involving them.

This technique, while not perfect, does reduce the number of times you must do page-access validation.   As long as the inbound request has a valid code, and the IP-address has not changed, and the page is not highly sensitive, you can get away with just the code-check.   (Naturally, you will use “hardened” session cookies, courtesy of many CPAN modules that will generate them for you, and you will verify that the connection uses HTTPS.)