22 November 2023 - 10.00pm
Our Network Admin team have continued to monitor things and the network has remained stable. The issue has been narrowed down and we will be doing some additional mitigations in the coming days (off peak) to ensure this issue will not happen again. What should have happend today is the network routes should have converged via BGP but this did not happen (as has happened occasionally in the past on the same network). The team have identified the issue and we will ensure we mitigate this as soon as possible. Any work will be completed off peak with a short period of network convergence causing a minor blip in connectivity lasting a few minutes maximum.
We take network uptime seriously and we do apologise for this outage today. As you will be aware we have had 100% network uptime in our BWF Northern Ireland data centre since it was opened nearly 3 years ago and we apologise for this 60 minute network outage today. It affected about one third of our client base hosted there.
22 November 2023 - 5.23pm
We are aware of a network outage that happened around 4pm on 22 November 2023 affecting about 33% of our client base. We lost connectivity on one of our main fibre lines into our data centre. We have multiple lines from various carriers so this in and of itself should not cause anything but a few minutes downtime. We do periodically test BGP failover so we are unsure why this did not happen today.
The network team were and still are working on this issue and at time of writing services are back on line. As the network team are still on site working we do not have a specific update but as soon as we get a report back from them we will update this announcement with more specific information.
We apologise for this blip in connectivity.
