Never heard of this "Kelko" thing before, but three clicks through their "About Us" page to their FAQ brings up the answer:

Q: Does Kelkoo search all shops on the web?
A: No, efficiently comparing prices from all shops on the web would be extremely difficult because there are far too many of them. Instead, we select a wide group of shops including big high street names and specialist internet shops. We are constantly looking for shops to add to our affiliate programme, and if we find a shop that has better offers than our current set, we contact them and try to include them on Kelkoo. If you can find a better price elsewhere, we'd love to hear it!

So they're more than likely writing scrapers for the sites they're specifically interested in, or they're probably big enough (as part of Yahoo) to have worked out some sort of arrangement with the source site to provide raw data.

Now there are approaches such as this Ruby work which provide a DSL (domain specific language) which lets you describe scrapers in DOM/CSS terms which make it easier to build up scrapers for new sites. I'm not aware of any Perl implementations of this idea, but that might steer you in the right direction.

In reply to Re^3: Crawler in perl by Fletch
in thread Crawler in perl by hodashirzad

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.