NODE-157 | Degraded Functionality
Resolved
Dec 18 at 02:37pm GMT
This has now been resolved. All the VPS are up and running. Thank you for your patience and understanding
Affected services
Updated
Dec 16 at 05:14pm GMT
We are continuing our urgent work on the affected node. We are currently in the process of securing all customer data to an external backup system.
To ensure absolute data integrity and prevent any potential corruption during this critical recovery phase, we are allowing the data transfer to proceed at a stable, controlled pace. We have decided not to rush this specific step to avoid complications.
The Plan Forward:
- Complete the data backup: We will wait for the full secure transfer to finish.
- Reprovision the node: We will rebuild the node.
- Restore services: We will migrate the data back and bring VPSs online.
We understand this is a severe incident, but our number one priority is the safety of your data. We are taking every precaution to preserve it, even if it requires extending the downtime.
Based on the current transfer speed, we anticipate this process will take several more hours. We will provide the next update as soon as the restoration phase begins.
Thank you for your patience as we prioritize the safety of your files.
Affected services
Created
Dec 14 at 04:50pm GMT
We are continuing recovery work on NODE-157 following an unexpected and complex failure.
The node is currently online; however, final restoration, validation, and data integrity checks are still in progress. While a full reinstall could have returned the node to service quickly, this was deliberately avoided to prevent permanent data loss. Our priority remains preserving customer data and restoring the node safely and correctly.
At this time, we estimate that recovery will be completed within the next 24 hours. Please note that this timeframe is an estimate, not a guarantee, due to the complexity of the issue encountered.
We recognize that this outage has exceeded normal expectations and understand the disruption this causes, particularly during a high-usage period. Work is ongoing, progress is being made, and updates will continue to be posted here as the situation develops.
Once recovery is fully completed, a post-incident report will be made available upon request.
We sincerely appreciate your patience while we complete this process.
Affected services