I think the key here is the use to which you put the extracted information. If it's for personal use then I wouldn't worry too much about it.

I wrestled a bit with a similar question -- I was distributing some modules that yanked information off of sites (historical stock quotes, to be precise). After some constructive conversations with pjf, I came to realize that scripts such as these are nothing more than a browser. The terms of service for a site apply to the user of the browser, not the author. So in this sense I passed the buck -- here's a tool, read the TOS of each site involved and see if it applies to *you* -- the TOS for the site does not apply to the tool in hand.

After all, what if you use mozilla and banish images from certain advert servers? It's not the authors of the browser's fault -- they merely provide a useful tool. The TOS of the site applies to the user of the browser.

As the user of your tool, you will have to examine how you are using the data you are fetching. If you're repackaging it or selling it as is, that's a problem ethically as well financially, potentially, if the information source comes after you. If, on the other hand, you are selling analytical work derived from the data, well, that's not so cut and dried since you're adding value -- as several people have pointed out, you should cut through the middle man and buy the information directly. Do this not to merely salve your conscience but to protect your legal liability.

But if it's for personal use then I think you're just fine and I wouldn't worry about it. You're using a modified browser, end of story.

Matt

In reply to Re: Fetching data from a corporate websites using LWP by mojotoad
in thread Fetching data from a corporate websites using LWP by Poblachtach32

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.