in reply to Re^4: Web scraping
in thread Web scrapping

While this page leaves a lot to be desired, the section Scraping Dynamic Websites does touch on the unreliability of basing parsing code on 'view source', since there's a great deal more to consider (previously mentioned domain specific knowledge, MDN Web Docs is a good starting point). I still automate scraping frequently, but only where there's no alternative sensible data source, see previous posts. If you can write it well, and once then this makes for a happier life than playing Whac-A-Mole with a sites such as this which frequently change their layout, html IDs and so fourth.