Server 22 Maintenance

  • Thursday, 20th July, 2017
  • 02:14am
A full report will be made available later today.  We'd ask you allow our staff time to work at issues to get sites back on line.  A summary will be regularly updated below with a time stamp.  Once all affected clients are restored a full reason for outage will be made available to clients.


Update: 7.00pm:  Restores complete.  The team are working through any failed accounts not and sorting manually
Update 3.00pm:  96% of accounts are back on line.
Update 1.00pm:  93% of accounts are back on line.  The rest will take a while longer as they are larger accounts.
Update 11.45am:  90% of accounts are back on line.  The largest accounts are still in progress.  Apologies if you are still affected.
Update 9.30am:  85% of sites are now back on line. Sites are being brought back on line in order of account size.  This means most clients will soon be back on line.  A small handful of massively large accounts will take a while longer.  
Update: 9.00am:  80% of sites are now back on line.  Working through issues on by one.  Update in an hour
Update: 7.30am:  65% of sites are now back on line.  Working through issues on by one.  Update in an hour
Update 6.30am:  50% of sites are now back on line.  Working through issues one by one.  Update in an hour
Update 5.39am:  We're waiting a report from the technicians in the data centre.  Precautionary restore ongoing just in case.
Update: 5am:  We're starting a precautionary restore to spare hardware.  RAID card replacement will happen within the hour.
Update 4.21am:  We're replacing the RAID card.  There is a 2 hour SLA for hardware replament in our data centre
Update 4am:  Apologies for the extended outage here on this server.  A senior admin will have an update soon
Update 3am:  We're still working on this please bear with us

Server 22 was showing a degraded array and one drive was missing.  We swapped the drive earlier.  The rebuild failed and the drive was showing as missing still.  As it's off peak UK we decided to take the server off line to re-seat all drives and re-seat teh RAID card and cables.


The array is currently rebuilding.  The server should be back within the hour.
« Back