in reply to Thwarting Screen Scrapers

I had hoped to limit this to the technical rather than philosophical points, but it looks like the replies are headed elsewhere.

How about an example close to home ?

I create a site called perldudes.com. Instead of developing my own community, I front-end perlmonks.com, taking the inbound http requests, pulling nodes from perlmonks, and substituting text as needed. ( s/perlmonks/perldudes/g, etc). I also put in my own advertisements and content, and maybe the interface is really crappy.

As for what this has to do with perl, It's obviously a bit off-topic. I am interested, however, in how any technique might be implmented in perl, and what modules might help me along.

I am aware that there is no way to completely stop this sort of thing. I'm looking for the best ideas to slow it down or at least stop the simplistic attempts

Abigail: I understand your points, but...If someone else can sell my product, but creates the whole customer selling experience, and I have to create, ship and support the product, how is that okay ? The "scrapers" go to great lengths to make sure the Customer doesn't see the fact that there are two parties involved. They also dish off support, etc, by front-ending the feedback forms.

Replies are listed 'Best First'.
Re: Re: Thwarting Screen Scrapers
by dwatson06 (Friar) on Jul 18, 2002 at 14:56 UTC
    kschwab,
    Find the ip it's coming from and block it. Unless it's client side scripting, it's going back to a central computer (or series of) somewhere.
    Block it.
    If you want to be really cool. When you find the IP, catch the POST or GET, go out to theonion.com or some other random site, do a get yourself and hand it to their request.
    The bad part about that is, you would be doing what they are but it would confuse the mess out of them for a moment ;o)
    Daniel
[reply]
Re: Thwarting Screen Scrapers
by Abigail-II (Bishop) on Jul 18, 2002 at 15:02 UTC
    I still don't see the big problem. If a person would go to your site and order something, you will have to create, ship and support the product. Just like you have to do when they go to someone elses site. Of course, if you don't want to create, ship and support a product, why do you have it?

    I do assume you are getting paid for creating, shipping and supporting the product. If not, and it's a burden to you, perhaps you should stop. ;-)

    What interests me is how they manage to get in the middle when it comes to paying. How are they getting their share? If they take a credit card number, take their share from the account, then pass the number to you so you take your part, the customers will frown, and someone will think "fraud".

    Abigail

[reply]
      It's not one situation, but many.

      Indeed, some of them do make their own charge on the credit card, and I end up handling the resulting mess.

      There's several variations on the theme, some of them actually calling out the correct name for the product, but acting like they are some sort of authorized reseller.

      Other ones have a relationship with vendors of similar products, and get paid for those purchases. They include my product only for completeness, and make no money on the transaction.

      They do, however, get the Customer eyeballs, and create confusion. My product gets tied in with their advertisements, or perhaps their interface keeps crapping out, and I get that feedback.

[reply]
      Part of the retail process is trying to get the customer to come back and give you more of their money. Hard to do that if they don't know who you are. 
      ()-()
 \"/
  `
[reply]

In Section Seekers of Perl Wisdom