Re: Perlmonks site has become far too slow
by Corion (Patriarch) on May 01, 2025 at 08:02 UTC
|
See Anonymous Google Chrome browsers now under additional scrutiny. We're currently serving/blocking 16 hits/s of bot crawlers, distributed across a lot of IP addresses which I suspect are rented residential IP addresses.
I don't think these crawlers run Javascript, and there is very little to do as we're blocking them as early as possible without moving behind Cloudflare or something like that.
| [reply] |
|
without moving behind Cloudflare or something like that.
I really hope it doesn't come to that. But if it does, please don't use Cloudflare. You are probably already aware of the reasons I say that but I'll be happy to elaborate (here or in private). Fastly seem to be doing a decent job protecting MetaCPAN and are clearly open to supporting Perl-related sites so they might be a decent candidate. Worth also talking to Pair if you haven't already. If they can't (or won't) keep the bad bots away from a site with their banner on it then that's not exactly a ringing endorsement of their services.
| [reply] |
|
From my side and POV, I think the number of users who disable javascript in their browsers, is small. I know it is an ideological statement and a security precaution, maybe, but it is not practical with current state of the internet, so perhaps not many people use it (i may be wrong of course, and in this case is the edge cases which matter the most!). I know that PerlMonks site prides itself on minimal or little javascript. Rightly so. After this disclaimer, can I dare suggest banning browsers which do not support javascript? That's towards the goal of preserving the site and its accessibility to humans, the big picture. Or, if browsers do not support javascript, redirect them somewhere else. Anyway, the site feels much better now, but as Co-Rion said, that's a rat race, and we lack the long tail.
| [reply] |
|
Since the crawlers have not signed up yet, all such changes would only affect Anonymous Monk.
See Anubis for something that could be implemented with relative ease. Instead of building a babel-tower of front-end proxies all needing a container to keep up, Anubis itself could be ported to a Perl function instead by somebody enterprising.
| [reply] |
|
I do. Well, at least selectively (I enable it when really needed). Mostly as a precaution, but also because I like to keep documents static when possible. As a bonus, it greatly reduces the number of requests. But sure, it's an ideological statement too.
return on_success() or die;
| [reply] |
|
Anecdotal but it does feel a little bit better past day or so (at least fewer outright page won't load instances). Thanks to those trying to keep things going at any rate. In the immortal words of Dr. Rumack, "It's a big building with patients, but that's not important right now.". Wait, that wasn't it . . . "I just want to tell you both good luck. We're all counting on you."
The cake is a lie.
The cake is a lie.
The cake is a lie.
| [reply] |
|
Perhaps, a next move, if needed, might be to use Anubis. I've seen some code hosting services (such as Codeberg and Sourcehut) using it and the protection seems good enough (though really a bit sad that it requires js to work, at least for now). It's a lot better (IMHO, at least) than resorting to Cloudflare.
It's also described here.
return on_success() or die;
| [reply] |
Re: Perlmonks site has become far too slow
by hippo (Archbishop) on May 01, 2025 at 09:52 UTC
|
This is going to push people away, including long time users of the site.
It has already done that. Just look at the drop-off in usage since this became an issue. The numbers here were on a gentle downward trend anyway but over the last 18 months or so it has had a serious effect.
Is this a bandwidth issue? A DB issue? An issue with an overwhelming number of bots swamping the site?
Mostly the last of these. Any large-scale, dynamic site will have been suffering the same. Every kiddie and his dog needs "training material" for their shiny new stochastic parrot. There is no silver bullet and techniques and success rates vary depending on the detail of the site under bombardment.
What I might suggest is to make use of the fact that the Monastery has multiple domains. Keep .org open and free and let the bots have at it (within reason). Put .com and net on another front-end host and there require a login. That way when the botalanche takes down host1 the genuiune users can still get in on host2. It's not great but it would be better than the status quo. Of course, if the bot activity is killing the back-end as well, then that's another matter.
| [reply] |
|
the Monastery has multiple domains. Keep .org open and free and let the bots have at it (within reason). Put .com and net on another front-end host
Multiple domains, yes. All the domains point to the same IP address. Behind that, we have some kind of "load balancing (*cough*)".
if the bot activity is killing the back-end as well
Of course, it is; PerlMonks is all back end.
Today's latest and greatest software contains tomorrow's zero day exploits .
| [reply] |
Re: Perlmonks site has become far too slow
by mvaline (Friar) on Aug 21, 2025 at 18:58 UTC
|
I'm glad you raised this point and I logged on today intending to post something similar. This remains an issue. I am averaging over 1 minute from click to page load. It was worse last night--I logged on with a friend because I wanted to share a reference to a great post from the past that explained something very well. Unfortunately, we had to give up because we couldn't get the home page to load.
Although I don't post much anymore, this site is important to me. If there is some way to contribute with time or resources to helping keep this great resource available, I'd like to help.
| [reply] |
Re: Perlmonks site has become far too slow
by kcott (Archbishop) on Aug 29, 2025 at 04:40 UTC
|
Unfortunately for me, PerlMonks has been pretty much unusable over the last week.
I normally connect with https://www.perlmonks.org/?node_id=861371 —
this takes me to my home node (but not logged in yet).
After persevering today, I see Last here: indicated "a week ago".
I had tried several times and, thinking I'd just been unlucky and the site was down, gave up.
Today I gave it some minutes; while still waiting, I pinged the org, com & net TLDs:
$ ping -c3 perlmonks.org
PING perlmonks.org (66.39.54.27): 56 data bytes
64 bytes from 66.39.54.27: icmp_seq=0 ttl=40 time=229.903 ms
64 bytes from 66.39.54.27: icmp_seq=1 ttl=40 time=230.685 ms
64 bytes from 66.39.54.27: icmp_seq=2 ttl=40 time=230.767 ms
--- perlmonks.org ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 229.903/230.452/230.767/0.390 ms
$ ping -c3 perlmonks.com
PING perlmonks.com (66.39.54.27): 56 data bytes
64 bytes from 66.39.54.27: icmp_seq=0 ttl=40 time=230.893 ms
64 bytes from 66.39.54.27: icmp_seq=1 ttl=40 time=230.677 ms
64 bytes from 66.39.54.27: icmp_seq=2 ttl=40 time=230.582 ms
--- perlmonks.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 230.582/230.717/230.893/0.130 ms
$ ping -c3 perlmonks.net
PING perlmonks.net (66.39.54.27): 56 data bytes
64 bytes from 66.39.54.27: icmp_seq=0 ttl=40 time=230.886 ms
64 bytes from 66.39.54.27: icmp_seq=1 ttl=40 time=229.992 ms
64 bytes from 66.39.54.27: icmp_seq=2 ttl=40 time=230.835 ms
--- perlmonks.net ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 229.992/230.571/230.886/0.410 ms
These looked reasonable.
After checking these results, I found I had finally connected to https://www.perlmonks.org/?node_id=861371.
I then proceeded to log in: this, again, took several minutes.
Subsequent attempts to access various (unrelated) nodes either took a long time or I got a "Connection timed-out" message.
I'm hoping I can post this; regardless, I won't be spending any more time on this site today —
I may give it another go over the weekend.
I'm using Firefox on MSWin10. I had updated earlier today to 142.0.1.
Problems over the last week or so would have used the last one or two versions.
| [reply] [d/l] [select] |
|
Yes, it has been particularly bad this week and as has been discussed in this and other similar threads it is driving genuine users away as a result. If there were an easy fix it would have been applied.
If you want to protect your favourite websites the solution is not to use any LLM-trained products or services whatsoever and to encourage everyone you know to take the same stance. We all need to cut off the revenue streams for these pariahs as it is the only thing which matters to them.
| [reply] |
|
If there were an easy fix it would have been applied.
The thing that puzzles me is that, of all the websites that I regularly peruse, perlmonks stands out (and I mean really stands out) as clearly being the slowest and flakiest.
Why is that ? What is the "feature" of perlmonks that makes it so extremely susceptible to these attacks ?
Cheers, Rob
| [reply] |
|
|
|
|
|
|
|
Thanks for all of the replies in the subthread that followed my post.
I logged in today (Sunday 31/8/25 ~11am AEST) without experiencing any delays.
I've been far less active on this site, over quite a few months, due to real life issues that I'm dealing with.
Waiting several minutes for pages to load is indeed frustrating; however, I won't stop visiting for this reason —
but, when slowness becomes a problem, I may cut my time here.
I will keep persevering. :-)
| [reply] |
Re: Perlmonks site has become far too slow
by Anonymous Monk on Aug 29, 2025 at 04:17 UTC
|
It's been really bad again the past week. Any updates? | [reply] |