Unexpected downtime / server upgrades

Well, it’s always frustrating when there is unplanned server downtime.

Today we upgraded the operating server operating system, and most of the server stack as well. This was expected to be a relatively brief outage after making backups of everything. Unfortunately the system upgrade took more than two hours, and another stretch of time for the 800+ packages involved. There are still a couple of sites offline awaiting non-essential manual over-rides.

For which I am deeply apologetic. I plan to address this over the next couple of weeks, developing an alternative model for system upgrades.