Like many are saying, I definitely believe it depends on the site you are retrieving the data from. If they don't have advertisements on their site, you hitting it every few hours with LWP is less stressful on their server then somebody hitting it more frequently with interactive browser. Even better, you could set up your script to only work at night, when few people would be there.

However, many sites have policies on this, which can often be found at the bottom of their site. For example, WhoWhere.com states the following in their terms of service:

(You agree not to) Sell, distribute, or make any commercial use of data obtained from any Lycos database or make any other use of data from any Lycos database in a manner which could be expected to offend the person for whom the data is relevant

-and-

Use automated means, including spiders, robots, crawlers, or the like to download data from any Lycos Network database.

Also, the terms of service for people.yahoo.com states:

You agree not to reproduce, duplicate, copy, sell, resell or exploit for any commercial purposes, any portion of the Service, use of the Service, or access to the Service.

The above statements make it sound like retrieving any data from either of those sites for any commercial purpose may be breaking their terms of service. So, I'd just make sure you read the terms of service and such for the site you're looking into. You may want to email them, and explicitly ask their permission -- they may let you do it, particularly if you tell them it'd only be once an hour throughout the night.

Good luck!
-Eric

--
Lucy: "What happens if you practice the piano for 20 years and then end up not being rich and famous?"
Schroeder: "The joy is in the playing."

In reply to Re: Fetching data from a corporate websites using LWP by andreychek
in thread Fetching data from a corporate websites using LWP by Poblachtach32

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.