Wow, then my mistake. I stand corrected. Thanks! I just saw so much JS noise and I searched on "638" in an attempt to find the actual price without a result, but that probably means I made a search mistake. Ooops! This does make the job quite a bit easier. This then now brings back up the "terms of use" and whether using this page is "legal" for the described usage. That I don't know.

I found this below: Geez, this page's code is a mess!: 

<div class="metal-title">
            Gold Price        </div>
<div class="nfprice">&#36;1,638.93</div>
<div class="table-variations">
<div class="single-variation-currency">

[download]
Update: I followed my own advice and googled "commodity api data". There appear to be lots of options. I haven't investigated nor do I endorse any of the sources. But one says "XXXX offers commodity prices data for almost 100 commodities, including gold prices, silver prices and oil prices from multiple sources. XXXX's simple API gives access to daily spot prices and historical commodity prices. That or other similar sounds promising.

The API for XXXX says a free user gets: "Authenticated users have a limit of 300 calls per 10 seconds, 2,000 calls per 10 minutes and a limit of 50,000 calls per day." Pay for users can go faster. This is much better than fiddling around with web page with fancy graphics. The data is returned in a format that is easy for computers to understand. Well geez as it should be if the "throttle" on a free account is an average of 30 requests per second!

Additional Update https://blog.quandl.com/getting-started-with-the-quandl-api This shows how to get the data you want in JSON or CSV files. The way to use Perl is to get this JSON data and do what you want with it. Look at https://docs.quandl.com/docs/in-depth-usage for some examples. Scraping a user web page is not the right way to get this info. Get the right API for the data that you need and then use Perl to just go crazy with this JSON, CSV or HTML data. Although Your Mother found the HTML representation of Gold Price on this initial page and yes parsing this page can get that number, it is not the "right way". Using an API to get the data you want is the "right way" and these API's are designed to be very performant. I mean geez, this API is designed so that you can hit it 50k times per day without even paying anything! If you need this data more often than that, you are into something much more advanced than your question indicates!

In reply to Re^3: how do I scrape this web page by Marshall
in thread how do I scrape this web page by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.