Connectivity issues for a limited number of customers nationwide. - Issue resolved
Hello,
The M247 NOC team have received a number of alerts nationwide affecting various services. These alerts have been triaged and major incident status has been triggered.
We are now in the process of performing initial diagnostics. Once these diagnostics have been completed, we will escalate through the necessary channels.
Further updates will be provided via the communication thread.
Next update time: 14:10 or sooner
Regards
M247 Support
Hello,
After engineers replaced the power modules, the device seemed to have stayed offline and seemed to have been stuck in the boot process.
However, After delays from datacentre engineers, infrastructure team was able to release the device from the boot process and bring it online, restoring services.
We can now confirm that all services affected due to this incident are restored. However, due to nature of the fault, we are monitoring the device to ensure it remains stable.
If you are still experiencing service loss or would like to request a PIR document, please contact ticket@m247.com.
Thank you for your patience throughout the incident. As part of our post incident review, we will conduct a root cause analysis and identify any relevant remediation activities.
Regards
M247 Support
Hello,
Please see our latest update on this incident below,
Two new power modules have been delivered to site and datacentre engineers are in the process of replacing the existing modules with the new modules.
As a final contingency measure, Field engineers are on the way to the datacentre with an estimated time of arrival of 23:30 to replace the network device affected.
Next update time: 22:00 or sooner
Regards
M247 Support
Hello,
Please see our latest update in regards to the incident below,
Replacement power modules have been sourced and will be delivered to the datacentre in approximately 30 minutes and will be installed by datacentre engineers.
As a last resort, Infrastructure engineers have also configured replacement hardware and field engineers are headed to the datacentre from Manchester with the replacement hardware kit.
Next update time: 20:30 or sooner
Regards
M247 Support
Hello,
Please see our latest update below,
After extensive troubleshooting and being unable to get the affected device online, Infrastructure engineers are now looking at sourcing replacement power modules locally to bring the connections back online as soon as possible.
We are also in the process of dispatching a field engineering team to visit the datacentre with further replacement inventory.
Next update time: 19:00 or sooner
Regards
M247 Support
Hello,
Please see our latest update below,
We have raised a remote assistance from the datacentre engineers. Infrastructure team is currently working with the engineers onsite to resolve the issue. Thus far, the issue seems to be due to local power issues on the hardware.
Engineers are physically reloading and reseating the affected modules.
Next update time: 17:30 or sooner
Regards
M247 Support
Hello,
Please see our latest update below,
Infrastructure engineers have remotely reset the affected hardware module on the network device, however the issue seems to persist. We have raised a request to access our device physically, but are currently facing delays in getting a response from the third party datacentre provider. We are actively following up with the provider.
Apologies for the inconvenience caused.
Next update time: 16:30 or sooner
Regards
M247 Support
Hello,
After performing initial diagnostics, we have determined a potential issue with one of our devices in London Datacentre.
We have now escalated this incident to our infrastructure team who will be coordinating the incident resolution works.
We have also raised remote diagnostics request with the datacentre and are awaiting their response.
Next update time: 15:10 or sooner
Regards
M247 Support