1 June 2021 - 7.30am
The node has remained stable overnight. There are some issues here we will address with priority (1) This server was deployed brand new in May 2020 but the hardware error log in iDRAC has 19 pages of concerns and the hardware took a full 2 hours to reboot yesterday while the boot sequence worked through the Dell standard hardware checks (the main cause of the delay). We are therefore having this server replaced today with a brand new machine and we will work with clients to move their data across with minimum disruption to them and (2) The root cause of the issue yesterday (extended by the long reboot time sadly) was a drive failing at 4.30pm yesterday. This failed drive caused the array to become unstable. When we physically pulled the drive out of the hot swap bay in the array the server stabilized and came back on line. We will hold off replacing that drive due to (1) above as we do not want to do anything to risk us needing to do another reboot.
31 May 2021 - 7.55pm
All servers are back on line on a degraded array. We will work with the data centre to have that failed drive replaced. It's a hot swap chassis so this should happen without any noticable performance issues.
31 May 2021 - 7.35pm
We had the data centre remote hands pull the failed drive and the array is no longer in Read Only Mode. Servers are booting up. We will keep providing some updates here and thank everyone for their extended patience.
31 May 2021 - 7.15pm
The node is back on line. We're unable to start any of the virtual machines. Our senior admins are working to remove the failed drive from the array and try again. We'll have another update on 30 minutes.
31 May 2021 - 6.30pm
Our system admin team have told us we should have some information within 30 minutes on the state of the server and the data. Please bear with us. We are doing our best to bring this server back on line. As always once we have the matter resolved we can give any client a full explanation but our priority now is to work to bring things back on online.
31 May 2021 - 6.00pm
Our system admin team are working still to bring the server back on line. We've determined that a drive failed in the array just before the server went off line and brought the server off line. Our senior system admin is now on shift and managing this issue. We do not have any more information at this time but please know this is being handled with top priority.
31 May 2021 - 5.00pm
VPS Node 518 has gone off line and our technical team are working hard to determine the issue and bring it back on line. We'll post updates to this page as we get them from our system administrator team. Our technical staff on live chat and support ticket will be available should you have any additional questions. We apologise for the outage. As it stands 17 VPS servers are off line. This issue is getting top priority from our support team.
