Re^4: Web scraping

its not AI generated

this is the reference where been learning how to web scrap and took so
+me examples from https://brightdata.com/blog/web-data/web-scraping-wi
+th-perl
[download]

you can see that my code is similar to the examples on that web

Comment on Re^4: Web scraping Download Code

Replies are listed 'Best First'.
Re^5: Web scraping by marto (Cardinal) on Jul 11, 2025 at 19:57 UTC
While this page leaves a lot to be desired, the section Scraping Dynamic Websites does touch on the unreliability of basing parsing code on 'view source', since there's a great deal more to consider (previously mentioned domain specific knowledge, MDN Web Docs is a good starting point). I still automate scraping frequently, but only where there's no alternative sensible data source, see previous posts. If you can write it well, and once then this makes for a happier life than playing Whac-A-Mole with a sites such as this which frequently change their layout, html IDs and so fourth.	[reply]

Replies are listed 'Best First'.

Re^5: Web scraping
by marto (Cardinal) on Jul 11, 2025 at 19:57 UTC

While this page leaves a lot to be desired, the section Scraping Dynamic Websites does touch on the unreliability of basing parsing code on 'view source', since there's a great deal more to consider (previously mentioned domain specific knowledge, MDN Web Docs is a good starting point). I still automate scraping frequently, but only where there's no alternative sensible data source, see previous posts. If you can write it well, and once then this makes for a happier life than playing Whac-A-Mole with a sites such as this which frequently change their layout, html IDs and so fourth.

[reply]