Re: On improving Perlmonks site availability

I havent seen this idea in your list, no idea whether it is viable:
What about providing a zipped download of the whole site, updated weekly or so?
Maybe stripped down from unnecessary/redundant info, so it is actually easier to use the download than scraping?

Comment on Re: On improving Perlmonks site availability

Replies are listed 'Best First'.
Re^2: On improving Perlmonks site availability by LanX (Saint) on Mar 30, 2026 at 23:00 UTC
If there was any "intelligence" behind the scraping, it would have long realized that we have only dozens of posts with new content per week. The AI business is hungry and greedily tries to swallow as much content as possible and many actors are throwing billions into training more bots. (Update: Moved the rest to Redirecting anonymous requests to offline copy on other servers) Cheers Rolf _{(addicted to the Perl Programming Language :) see Wikisyntax for the Monastery}	[reply]

Replies are listed 'Best First'.

Re^2: On improving Perlmonks site availability
by LanX (Saint) on Mar 30, 2026 at 23:00 UTC

The AI business is hungry and greedily tries to swallow as much content as possible and many actors are throwing billions into training more bots.

(Update: Moved the rest to Redirecting anonymous requests to offline copy on other servers)

Cheers Rolf
_{(addicted to the Perl Programming Language :)

see Wikisyntax for the Monastery}

[reply]