in reply to Sessions & Search Engines

Figure out what the different search engine crawlers use for a user agent, then configure your web server to skip the language page for those user agent strings.

Replies are listed 'Best First'.
Re^2: Sessions & Search Engines
by ahmad (Hermit) on Apr 05, 2010 at 21:31 UTC

    I've had this idea, but this way I cannot make it crawl the content in multiple languages, I will have to set a default language only.

[reply]

In Section Seekers of Perl Wisdom