www.ably.io
Back

Incident Details

All times are shown in UTC

28th July 2020 08:25:00 AM

Sporadic high latencies in us-east-1

We are investigating intermittent high latencies in REST requests to us-east-1. Other datacenters are unaffected.

28th Jul 09:20 AM

We have temporarily redirected traffic away from us-east-1 to other datacenters

28th Jul 09:35 AM

The issue is due to AWS issues with resolving dns from within us-east-1: https://status.aws.amazon.com/ . We are leaving traffic redirected away from us-east-1 until that is resolved. (Any connections that were already connected to us-east-1 will remain for now)

28th Jul 11:41 AM

Following AWS reporting the dns resolution latency issues as fixed on their end, which we have confirmed, we are now re-enabling traffic to us-east-1.

Total customer-visible effect should have been very little, other than occasional latency spikes for requests to us-east-1 in the time between the AWS issue beginning and when we redirected traffic away from that region at 08:15 UTC, and fractionally higher latencies for customers near us-east-1 who were redirected to us-west-1 for the duration.

Resolved

in about 3 hours