Sessions & Search Engines

ahmad has asked for the wisdom of the Perl Monks concerning the following question:

Hello

In order to save language choice, I have used sessions CGI::Session module.

I present a 'Choose your language' page when you first enter the website then the user will be able to surf the website in his/her own language.

This has resulted in inability for search engines to spider the pages behind language choice page (they don't accept sessions/cookies).

The only solution that comes into my mind is keep passing ?lang=[lang] for every single page/action which will result in re-writing too many links and requires modification in too many files

Do you have a better solution??

Comment on Sessions & Search Engines Download Code

Replies are listed 'Best First'.
Re: Sessions & Search Engines by Corion (Patriarch) on Apr 05, 2010 at 18:05 UTC
My proposed solution would be to not have such a "landing page" blocking me (or crawlers) from visiting the site at all. Look at how Youtube does it - they add an (annoying) element to the page asking for whether I would like to "fix" my location. I surf with cookies and Javascript disabled by default, so unless your site was the only site returned in a search and immediately relevant to my interests, I'd never go beyond your "Choose your language" thing.	[reply]
Re: Sessions & Search Engines by Herkum (Parson) on Apr 05, 2010 at 21:04 UTC
Why are you not setting a default language when the user has not specified any language? Depending on how clever you are you can set a default depending on where the user is coming from.	[reply]
Re^2: Sessions & Search Engines by ahmad (Hermit) on Apr 05, 2010 at 21:29 UTC
Yes I can, but this way search engines like google will only crawl the English version which is NOT what I want.	[reply]
Re^3: Sessions & Search Engines by Herkum (Parson) on Apr 06, 2010 at 02:50 UTC
Perhaps you should look at this, http://googlewebmastercentral.blogspot.com and the section "Crawling and indexing your multilingual website" which tells you how to get the site crawled in several languages.	[reply]
Re^4: Sessions & Search Engines by ahmad (Hermit) on Apr 06, 2010 at 16:33 UTC
Re: Sessions & Search Engines by lostjimmy (Chaplain) on Apr 05, 2010 at 18:22 UTC
Figure out what the different search engine crawlers use for a user agent, then configure your web server to skip the language page for those user agent strings.	[reply]
Re^2: Sessions & Search Engines by ahmad (Hermit) on Apr 05, 2010 at 21:31 UTC
I've had this idea, but this way I cannot make it crawl the content in multiple languages, I will have to set a default language only.	[reply]