the HTTP_REFERRER contains the URL of the page that the user had on their browser when the current request is made. So, when you are on the page www.site.com and click the link to go to www.domain.com, at that point www.site.com is the HTTP_REFERRER.
Then you're looking at the page at www.domain.com, and you click the image. So www.domain.com is the HTTP_REFERRER. If you need the referrers referrer, you'll need to pass that info along when www.domain.com is served up.

cookie, cgi hidden field, i dont know exactly what you're doing, but there should be some storage mechanism that suits your needs.