Finding dates from web pages

cormanaz has asked for the wisdom of the Perl Monks concerning the following question:

Greetings monks. I have a bunch of URLs of news articles and need to get the publication dates from these, if available. There is a python library designed especially for this purpose. I'm wondering if there is any similar Perl module. I've searched around and the only thing I found was Web::Scraper which would take quite a bit of rules development to do the job. Am hoping maybe someone has done that work already.

Comment on Finding dates from web pages

Replies are listed 'Best First'.
Re: Finding dates from web pages by Corion (Patriarch) on Feb 24, 2020 at 20:22 UTC
I also know of another Python library, article-date-extractor, which has a set of regular expressions. I haven't ported it to Perl though.	[reply]
Re: Finding dates from web pages by talexb (Chancellor) on Feb 25, 2020 at 13:18 UTC
Can you get what you want just from doing a HEAD on the web page? That would give you the Last Updated date, I think. I'm not sure if that's exactly what you want. Alex / talexb / Toronto Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.	[reply]
Re^2: Finding dates from web pages by cormanaz (Deacon) on Feb 25, 2020 at 20:39 UTC
That works for some pages that have known meta fields like pubdate or time, but many web pages don't use them. I think the Python library applies some heuristics in such cases.	[reply]
Re^3: Finding dates from web pages by marto (Cardinal) on Feb 25, 2020 at 20:53 UTC
"I think the Python library applies some heuristics in such cases" You could look at the Python code and implement the same thing in perl. Let me know if you get stuck.	[reply]
Re^4: Finding dates from web pages by cormanaz (Deacon) on Feb 26, 2020 at 15:22 UTC
Re^5: Finding dates from web pages by marto (Cardinal) on Feb 26, 2020 at 15:29 UTC
A reply falls below the community's threshold of quality. You may see it by logging in.