Hi,

We are having problems with a site where session ID's are being set, despite the fact that the PHP code is meant to exclude the session ID from the link/url. This is of course causing some grief to the website owner, as we have seen both Yahoo and MSN with links to the website, and they contain the session ID. This can potentially cause big probems.

We have reviewed what needs to be done, and need to test the new PHP code, however do not want to use any of the "Search Engine Simulators" around, I would rather we place a Perl script on the website, and do our own isolated testing.

So, does anyone know of a good Perl script that simulates a 'spider crawl' please, just to show the links and related links, so that we can thouroughly test that session ID's are not appearing. If possible, the script would allow us to enter the 'user agent', because we only want to turn sessions off, for spiders/bots,etc, not the general public.

The type of 'results' I needed are the same as produced by these simulators:

http://www.1-hit.com/all-in-one/tool.search-engine-viewer.htm

http://www.webconfs.com/search-engine-spider-simulator.php

Thanks,

Peter

In reply to Search Engine Simulator by peterr

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.