XEN35 Node Degraded RAID Array
-
Sunday, 16th July, 2017
-
17:46pm
Update 12.21am: Rebuild is complete - array is optimal. Zero impact to website uptime.
Update 12.05am: Rebuild Progress on Device at Enclosure 252, Slot 1 Completed 91% in 90 Minutes.
Update 10.30pm: Drive replaced and RAID array is rebuilding. 7% completed so far. Zero impact to website uptime
Update 10.25pm: Drive Replaced - Hot Swap with no downtime RAID rebuilding
Our monitoring system has shown us that the RAID Array on Xen35 Node is degraded. One drive has fallen out of the array and is showing as missing. We need to change this drive as soon as possible and rebuild the array. The server has a hot swap chassis and at 10pm on Sunday 16 July 2017 we'll be replacing the failed drive. After we replace the drive the array will rebuild again until it is optimal.
During rebuild there will be higher than normal IO (Input Output) wait time on the array but we're not anticipating any significant issues. The rebuild should complete overnight and by morning everything will be optimal.
We'll start the backups early this evening (7pm ) so we have up to date backups just as an additional precaution but nothing at all indicates this will be anything other than routine.
