I know I said recently, several times, that PerlMonks was easily accessible after several attempts to fix it, and many honest thank yous to Corion and everyone involved. But it seems to me that as it is now, it has degraded since then. OTOH it is much much better than when the problem peaked (and my current post had no timeout unlike other recent posts). But still it times-out quite often. Perhaps it is what I said before that if indeed the problem is caused by (decentralised) bots then they will take all available space if it is given to them by these improvements. So perhaps this is what we see, bots upping up the tempo because they found extra harvesting-bandwidth which PerlMonks itself has given them. If true, then perhaps the route of hacking the content-delivery method of the site can become soon a dead-end.

I also mentioned that PerlMonks could do with a quality-of-service monitor, as simple as recording access time of some few random nodes every few minutes. In order to assess various solutions, the problem must be quantified. This is paramount and, talking from the comfort of my keyboard, sounds easy. I think some fellow Monks here were already keeping such a record. Well, let's install this internally ASAP. EDIT: thanks to hippo I have learned that there is such QoS monitor already running, see Re^2: Still excessively slow, and graphically here https://www.perlconsultancy.com/perlmonkserrors.shtml

One added side-effect or THE main-effect is this: Searching on search-engines for something Perl-related often matches results from PerlMonks and at the very top. Which is great! Alas, I, subconsciously, avoid them or if I click on them, it is often that I have my problem solved before they open in my browser, OK exaggerating :). The gist is that the site also looses the status it had as "finding *existing* answers" and the vibe on the 'net is basically :(((( because we now have to access the &$^$&#@& Overflow which is steadily goose-stepping into fascism-land and has really become an uncomfortable experience for free-spirited minds. I wish I could avoid it, at least for Perl.

I wonder what happens when sites infested by these decentralised bots, i.e. those emanating from random, personal devices and IPs etc., are hosting advertisements. Do these sites see an exponential increase in ad-related revenues because of the bot-attacks? If the access-related problems we see with PerlMonks can be solved with money, then perhaps do consider joining the dark site? !Heresis! I know.

Apropos heresis (OT), I propose a new XP level: Heretic. It can be even on top of Pope.

thanks for all the fish, bw, bliako

I also mentioned that PerlMonks could do with a quality-of-service monitor, as simple as recording access time of some few random nodes every few minutes. In order to assess various solutions, the problem must be quantified. This is paramount and, talking from the comfort of my keyboard, sounds easy. I think some fellow Monks here were already keeping such a record. Well, let's install this internally ASAP.

That was already done quite some time ago. The on-site record is here.

---

🦛

> The on-site record is here.

FWIW: This page has an orphaned <td align= breaking the overall table design (the nodelets jump to the bottom) and I don't know how to patch it.

update

OK seems to be a side product from chatterbot which has an overflow problem:

            <tr>
                <td>2026-01-16 17:00:00</td>
                <td align="right">319</td>
                <td align="right">0.527</td>
                <td align="right">0.683</td>
   
### CUT OFF HERE

<!-- END general container -->
[download]

Cheers Rolf
_{(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery}

oh thanks, that's news to me. It has good hours and bad hours.

I know I said recently, several times, that PerlMonks was easily accessible after several attempts to fix it, and many honest thank yous to Corion and everyone involved. But it seems to me that as it is now, it has degraded since then. OTOH it is much much better than when the problem peaked (and my current post had no timeout unlike other recent posts). But still it times-out quite often. Perhaps it is what I said before that if indeed the problem is caused by (decentralised) bots then they will take all available space if it is given to them by these improvements. So perhaps this is what we see, bots upping up the tempo because they found extra harvesting-bandwidth which PerlMonks itself has given them.

I understand that Perlmonk's version of the Everything Engine is a big hurdle in changing the Perlmonks code, and that it is hard to harden it against DDoS. I've accidentally hidden a - hopefully simple - proposal to keep the unwanted, misbehaving bots away from the EE in Re^2: Taking advantage of E2 improvements.

It boils down to avoiding passing bot requests to the EE by becoming even more unfriendly to anonymous users than Perlmonks is now.

• Prevent all anonymous access to the Perlmonks code running on the EE, except for well-known, well-behaved bots (e.g. Google), right in the webserver, before any Perl code is executed. Require in the webserver that the request contains a valid login/session cookie, or comes from well-known IP addresses of well-behaved bots. In any other case, redirect to a static HTML document with a login form. Yes, this effectively locks out anonymous users. Put a hint on the login page to either create an account or use Google's cache.
• Add a new CGI/FastCGI/modperl/anything-non-EE program that can check username and password from the static login form against the database used by the EE. Have that program set the login/session cookie, compatible with EE, and redirect to the Monasty Gates on success. On error, redirect back to the static login page.
• Add a second static HTML document with a new user form, querying just username, password and email address.
• Add a second CGI/FastCGI/modperl/anything-non-EE program that can inject a new user into the EE database, set the login/session cookie, compatible with EE, and redirect to the Monasty Gates on success. On error, redirect back to the static create user page.
• Both programs may need some bot lock-out mechanism. Captcha, some client-side Javascript, IP-based rate limiting, whatever helps. And because this is outside the EE, changing the bot lock-out should be way easier than within the EE.

Again, I'm aware that this is not friendly to anonymous users. But if Perlmonks keeps being DDoS-ed by bots, it won't matter, because nobody will be here any more.

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Again, I'm aware that this is not friendly to anonymous users. But if Perlmonks keeps being DDoS-ed by bots, it won't matter, because nobody will be here any more.

I agree with afoken. No one will come to this site when they have to wait minutes (sometimes many minutes) between requests.

"It's not how hard you work, it's how much you get done."

> Prevent all anonymous access to the Perlmonks code running on the EE

I think that summarizes what I suggested here Re^8: Perlmonks site has become far too slow (Reverse Proxies).

Using nginx reverse proxies with rules to only pass thru logged in users (or yet uncached requests) and alternatively serving static content to anonymous clients (or blocking them if needed).

The EE can flag updated content such that nginx would update it's cache automatically.¹

Looks like an out-of-the box solution for me which can easily scale.

Nginx would grow it's cache gradually and carry the bot load while keeping the logged in users operational.

Cheers Rolf
_{(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery}

¹) That's the only change we would need at the backend which can be implemented and improved gradually.

> It boils down to avoiding passing bot requests to the EE by becoming even more unfriendly to anonymous users than Perlmonks is now.

No need to ruin the perlmonks to save perlmonks! Anubis saved nitter and it can save perlmonks too! C'mon guys it's not that hard...

Actually I have had the experience that over the past two weeks, while there still have been several periods where the site is slow, when the site is reachable, it's actually fast and usable, which is a big change from the way it was in the months before - before, it was basically excessively slow all the time. So, I find the current situation an improvement...

The site was slow until a few days ago, but lately (including today) it popped right up. Current time is just before 1900 UTC.

According to pm server stats, there error rate was 100% 3 hours ago.
map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

> According to pm server stats, there error rate was 100% 3 hours ago.

This is a denial of service attack, a serious state and federal crime in the United States. Report the offenders to the FBI. They could go to jail for up to 10 years and be liable for millions in damages.

So probably try in your NZ afternoon or evening?

(I know it's not a good solution and not what you asked for, but well ... beggars can't be choosers)

I have good experience at my night time too, but sometimes during the day as well. The key is to be persistent as hell and try way more than once. Why can't you guys solve this problem? Keeping bots off a website is trivial! Block the entire offending ASN if necessary. Or use something like anubis:
Anubis is a Web AI Firewall Utility that weighs the soul of your connection using one or more challenges in order to protect upstream resources from scraper bots. This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies. Anubis is as lightweight as possible to ensure that everyone can afford to protect the communities closest to them. Anubis is a bit of a nuclear response.

github.com/TecharoHQ/anubis

As of now, an AM can't ask a question ("Post a new question!" leads nowhere), and SoPW page seems broken, to begin with (can't browse to "Older entries", etc.)

confirmed, but i think it's only for Seekers of Perl Wisdom. i wonder if the readmore tag on the last entry is the cause.

I think you're right in that it is something to do with Warn if STDIN pipe is missing or unwanted. When viewing SoPW anonymously, the nodelets only start after the header for this post and all the subsequent posts and the new question form are missing. The <!-- END general container --> tag is present immediately after the closing div of the readmore tag which explains the broken layout.

This looks to be a bug (it's reproducible) rather than a direct symptom of heavy load.

---

🦛