Offline Copy of perlmonks

UPDATE

Never mind, I found a way to throttle wget and bend it to my needs :-)

I'd like to locally mirror this site for personal use. Need at least access to my thought processes of the last 2 decades.

Problem is that I don't want to put even more strain on the servers.

I looked at the wayback machine as alternative source and unsurprisingly it had problems accessing threads since "the AI" started its distributed denial of service attack.

I suppose a compressed tar ball would be in the range of 0.1 to 1.0 GB and could be offered as a torrent. (Unfortunately AI wouldn't be "I" enough to use that instead)

Hmm ... Alternatively to plain html maybe only the XML versions of the posts plus a mojolicious app running the search engine and rendering the posts.

Parts of this post rather belong to meditation, feel free to move it if appropriate.

Cheers Rolf
_{(addicted to the Perl Programming Language :)

see Wikisyntax for the Monastery}

Comment on Offline Copy of perlmonks

Replies are listed 'Best First'.
Re: Offline Copy of perlmonks by Corion (Patriarch) on Mar 28, 2026 at 06:40 UTC
Note that the XML is also mirrored at `https://corion.net/perlmonks/nnnnn.xml`, which likely has better performance than the site itself.	[reply] [d/l]
Re^2: Offline Copy of perlmonks by LanX (Saint) on Mar 29, 2026 at 21:51 UTC
Thank you very much, even if they only start in 2007. And I suppose they don't reflect edit-updates by the users. But it's a good start! :-) Sorry for hammering your server with 700k requests. It might be a good idea to tar -z bundles.tgz in the same number range to avoid excessive traffic. Cheers Rolf _{(addicted to the Perl Programming Language :) see Wikisyntax for the Monastery}	[reply]
Re: Offline Copy of perlmonks by Fletch (Bishop) on Mar 27, 2026 at 14:30 UTC
That's actually an interesting idea (a local mirror of the XML versions). Can any knowledgeable people comment with (ballpark even) how large the overall backup of the site is / might be? The cake is a lie. The cake is a lie. The cake is a lie.	[reply]
Re^2: Offline Copy of perlmonks by LanX (Saint) on Mar 27, 2026 at 22:05 UTC
About 1 GB ą 50% uncompressed text. Rationale: ~ 1 million nodes at 1 kB average XML) FWIW: Using the XML for alternative client is an idea as old as the site... It's mentioned somewhere in the doc catacombs. Cheers Rolf _{(addicted to the Perl Programming Language :) see Wikisyntax for the Monastery}	[reply]