How to index dynamic page?

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Does anyone know how search engines index dynamic pages so that they can find a page "http://sitename/path/scriptname/querystring" directly?

Thanks.

Comment on How to index dynamic page?

Replies are listed 'Best First'.
Re: How to index dynamic page? by thraxil (Prior) on Aug 17, 2001 at 03:02 UTC
a lot of search engines ignore dynamic pages and just chop off everything after a '?'. this has lead to lots of workarounds using things like mod_rewrite to change the url to something that doesn't look as dynamic. as far as how the search engines which do handle dynamic pages (such as google) store the url internally, i'd guess that they just store it as text, query string and all. at least, that's how i did it when i wrote an mp3 spider for a music site and i didn't encounter any problems doing it that way. anders pearson	[reply]

Replies are listed 'Best First'.

Re: How to index dynamic page?
by thraxil (Prior) on Aug 17, 2001 at 03:02 UTC

a lot of search engines ignore dynamic pages and just chop off everything after a '?'. this has lead to lots of workarounds using things like mod_rewrite to change the url to something that doesn't look as dynamic.

as far as how the search engines which do handle dynamic pages (such as google) store the url internally, i'd guess that they just store it as text, query string and all. at least, that's how i did it when i wrote an mp3 spider for a music site and i didn't encounter any problems doing it that way.

anders pearson

[reply]