Back to overview

Unexpected Service Interruption

May 6, 2026 at 4:20am UTC
Affected services
Uptime

Resolved
May 6, 2026 at 4:20am UTC

Status: Resolved

We experienced an unexpected service interruption affecting the platform.

Incident Summary:
The outage was triggered by an unplanned, hardware-level reboot of our primary VPS instance within the Oracle Cloud infrastructure. This caused a temporary loss of connectivity and prevented the automatic restart of core services.

Timeline of Events (UTC):
03:52 UTC: Instance initiated an unplanned reboot / infrastructure migration.
04:15 UTC: Manual recovery initiated; PM2 process managers restored.
04:20 UTC: All services confirmed online and operational.

Post-Mortem & Prevention:
Analysis confirmed that a system-level permission conflict prevented our automated recovery scripts from executing immediately post-reboot. We have implemented the following fixes:
Permission Patch: Updated systemd unit configurations to bypass SELinux execution blocks.
Buffer Optimization: Allocated additional swap memory to handle future resource spikes and prevent kernel panics.

We apologize for the disruption and are monitoring the system for continued stability.