Start Time: Thursday, October 28, 2021, at 16:56:52 UTC
End Time: Thursday, October 28, 2021, at 22:17:24 UTC
Duration: 5:20:32
What happened:
Email notifications of all kinds, including from alerts, were delayed for about 5 hours. Notifications sent by Slack and Webhooks were not affected.
Why it happened:
Our email service provider (Sparkpost) experienced an incident that caused delays for all emails from the LogDNA service. We rely on this service to deliver email of all kinds, including notifications for alerts. Email messages were delayed and queued until our email service provider was able to recover.
More information on the incident can be found at Sparkpost’s Status Page: https://status.sparkpost.com/incidents/bwl8dr6gwmts?u=ydzrh5x205pf
How we fixed it:
No remedial action was possible by LogDNA. We waited until the incident from Sparkpost, our email hosting provider, was resolved.
What we are doing to prevent it from happening again:
For this type of incident, LogDNA cannot take proactive preventive measures.