Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Site outage

by Co-Rion (Monk)
on Dec 08, 2021 at 21:35 UTC ( #11139479=monkdiscuss: print w/replies, xml ) Need Help??

We had an intermittent outage, but now the site is back up.

The outage was caused by an upgrade to the MySQL client libraries, which made them not want to talk to the server anymore. The error code was ERROR 2027 (HY000): Malformed packet. The solution is described on StackOverflow.

The resolution was to upgrade the DB password on the server to a new algorithm, which was a fix done in three minutes.

The hard parts were getting home, looking through all the logs, and finding the error :)

Replies are listed 'Best First'.
Re: Site outage
by jdporter (Chancellor) on Dec 09, 2021 at 01:40 UTC

    Please don't ever leave! I can't do what you do!

    I reckon we are the only monastery ever to have a dungeon staffed with 16,000 zombies.
Re: Site outage
by stevieb (Canon) on Dec 08, 2021 at 22:13 UTC

    Good work, and thanks to your prompt attention to the matter and the speedy resolution!

Re: Site outage
by LanX (Sage) on Dec 09, 2021 at 11:46 UTC
    > which made them not want to talk to the server anymore.

    curiously this was not continuously so.

    It was sometimes possible to interact, I even posted during the "outage", tho it took me dozens of attempts.

    Thanks for fixing, anyway! :)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

      Maybe only some of the servers were misbehaving?

      map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
Re: Site outage
by GotToBTru (Prior) on Dec 10, 2021 at 19:56 UTC
    The outage was caused by an upgrade ...

    Reminds me of the pilot's adage that at no time is a plane more likely to be unsafe than just after it's been fixed.

    But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Romans 5:8 (NASB)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: monkdiscuss [id://11139479]
Front-paged by haukex
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2022-05-25 20:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (90 votes). Check out past polls.

    Notices?