Ingestion delay in app.scalyr.com
Incident Report for DataSet
Resolved
This incident has been resolved.
Posted Oct 24, 2024 - 23:49 UTC
Monitoring
The UI should now be loading as expected. We have increased the database connection limit to accommodate more concurrent connections. The queue is in the process of recovering and is gradually processing the backlog of events.
Posted Oct 24, 2024 - 19:11 UTC
Update
The aggressive scaling out of servers led to a 500 error when loading the page due to hitting the database connection limit. We are currently in the process of scaling the servers back in, and the error should be resolved shortly.
Posted Oct 24, 2024 - 18:11 UTC
Identified
A misconfiguration deployed this morning prevented the servers from scaling up correctly. We are currently in the process of manually scaling up the servers to manage the ingestion volume effectively.
Posted Oct 24, 2024 - 17:13 UTC
Investigating
We are currently investigating the issue.
Posted Oct 24, 2024 - 16:54 UTC