Unexpected Service Interruption
Resolved
May 6, 2026 at 4:20am UTC
Status: Resolved
We experienced an unexpected service interruption affecting the platform.
Incident Summary:
The outage was triggered by an unplanned, hardware-level reboot of our primary VPS instance within the Oracle Cloud infrastructure. This caused a temporary loss of connectivity and prevented the automatic restart of core services.
Timeline of Events (UTC):
• 03:52 UTC: Instance initiated an unplanned reboot / infrastructure migration.
• 04:15 UTC: Manual recovery initiated; PM2 process managers restored.
• 04:20 UTC: All services confirmed online and operational.
Post-Mortem & Prevention:
Analysis confirmed that a system-level permission conflict prevented our automated recovery scripts from executing immediately post-reboot. We have implemented the following fixes:
• Permission Patch: Updated systemd unit configurations to bypass SELinux execution blocks.
• Buffer Optimization: Allocated additional swap memory to handle future resource spikes and prevent kernel panics.
We apologize for the disruption and are monitoring the system for continued stability.
Affected services