Never heard of this "Kelko" thing before, but three clicks through their "About Us" page to their FAQ brings up the answer:
Q: Does Kelkoo search all shops on the web?
A: No, efficiently comparing prices from all shops on the web would be extremely difficult because there are far too many of them. Instead, we select a wide group of shops including big high street names and specialist internet shops. We are constantly looking for shops to add to our affiliate programme, and if we find a shop that has better offers than our current set, we contact them and try to include them on Kelkoo. If you can find a better price elsewhere, we'd love to hear it!
So they're more than likely writing scrapers for the sites they're specifically interested in, or they're probably big enough (as part of Yahoo) to have worked out some sort of arrangement with the source site to provide raw data.
Now there are approaches such as this Ruby work which provide a DSL (domain specific language) which lets you describe scrapers in DOM/CSS terms which make it easier to build up scrapers for new sites. I'm not aware of any Perl implementations of this idea, but that might steer you in the right direction.
| [reply] |
| [reply] |
No ... not at this moment. Maybe if the Semantic Web ever takes off but I'm not holding my breath on that happening.
| [reply] |