You say that you "checks the referrer". You know that the HTTP Header "Referer" can be set to any arbitary value by the client? Do not trust any data sent from the client, including the "Referer" header.

Encrypting the session ID is nonsense. It does not improve security. Just make sure you do not have predictable session IDs. Use long random values (or UUIDs).

Alexander