www.ably.io
Back

Incident log archive

All times are shown in UTC

January 2016

15th January 2016 07:59:39 AM

Website tokens for demos and new sign ups failing

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

15th Jan 10:18 AM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

15th Jan 01:12 PM

Due to an upgrade to the realtime system some operations on the website were failing for a short period. The following website services were effected:
- Tokens used for demos and documentation examples
- New sign ups on the website

Resolved

almost 3 years ago
8th January 2016 11:02:43 PM

ably.io website caching issues

One of our Redis servers used for caching was unavailable for a period of 10 minutes resulting in intermittent issues with the website.

8th Jan 11:25 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

almost 3 years ago
8th January 2016 09:11:32 AM

ably.io website cacheing issues

One of our Redis servers used for cacheing was unavailable for a period of 10 minutes resulting in intermittent issues with the website.

8th Jan 09:11 AM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

8th Jan 09:37 AM

The issue was resolved automatically by our health services

Resolved

almost 3 years ago

December 2015

15th December 2015 01:49:14 PM

Our automated health check system has reported an issue in realtime South America

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

15th Dec 01:50 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

almost 3 years ago
15th December 2015 11:23:14 AM

Our automated health check system has reported an issue in realtime South America

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

15th Dec 11:25 AM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

almost 3 years ago
12th December 2015 07:51:28 AM

Our automated health check system has reported an issue in realtime South America

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

12th Dec 07:52 AM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

almost 3 years ago
11th December 2015 09:26:29 PM

Our automated health check system has reported an issue in realtime South America

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

11th Dec 09:27 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

almost 3 years ago

November 2015

21st November 2015 09:28:05 PM

Minor outage in Europe Frankfurt

There was a minor outage in Europe Frankfurt for a few minutes during an upgrade. All other regions were operating correctly so traffic would have been routed to those regions.

21st Nov 09:29 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

almost 3 years ago
21st November 2015 09:27:05 AM

Our automated health check system has reported an issue in realtime Europe Frankfurt

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

21st Nov 09:28 AM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

almost 3 years ago
13th November 2015 12:44:28 AM

Our automated health check system has reported an issue in realtime South America

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

13th Nov 12:46 AM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

13th Nov 12:56 AM

There was a short loss of connectivity to this region with little or no disruption to realtime clients.

Resolved

almost 3 years ago
5th November 2015 10:52:29 AM

Our automated health check system has reported an issue in website ably.io website

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

5th Nov 10:58 AM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

5th Nov 11:04 AM

The Heroku Postgres service was down for a brief period causing some downtime for the Ably website. We are looking at an upgrade now to help mitigate this type of issue in future

Resolved

almost 3 years ago

October 2015

28th October 2015 10:13:26 PM

Our automated health check system has reported an issue in realtime Asia Singapore

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

28th Oct 10:16 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

Resolved

about 3 years ago
15th October 2015 11:31:19 PM

Our automated health check system has reported an issue in realtime Asia Singapore

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

15th Oct 11:31 PM

An additional fault has been detected in realtime US West Oregon

15th Oct 11:31 PM

An additional fault has been detected in realtime Europe Frankfurt

15th Oct 11:31 PM

An additional fault has been detected in realtime South America

15th Oct 11:31 PM

An additional fault has been detected in realtime US West California

15th Oct 11:31 PM

An additional fault has been detected in realtime Australia

15th Oct 11:33 PM

An additional fault has been detected in realtime Europe Ireland

15th Oct 11:48 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

16th Oct 10:53 AM

The majority of regions were affected by a faulty deployment. During this time at least one region was up and running at any one time, so service continuity should have been maintained, albeit at reduced performance. We are currently integrating a change to our deployment process to prevent this happening again.

Resolved

about 3 years ago
13th October 2015 01:11:40 PM

Our automated health check system has reported an issue in website ably.io website

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

13th Oct 01:12 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

13th Oct 02:40 PM

Some website features were effected by an incident in the realtime system related to a cluster state fault. We are investigating the root cause, but in the mean time, services have been resumed.

Resolved

about 3 years ago
13th October 2015 12:47:47 PM

Our automated health check system has reported an issue in realtime Australia

This incident was created automatically by our automated health check system as it has identified a fault. We are now looking into this issue.

13th Oct 12:49 PM

Our health check system has reported this issue as resolved.
We will continue to investigate the issue and will update this incident shortly.

13th Oct 01:07 PM

An additional fault has been detected in realtime US West Oregon

13th Oct 01:11 PM

An additional fault has been detected in realtime US West California

13th Oct 01:12 PM

An additional fault has been detected in realtime US East Virginia

13th Oct 01:12 PM

An additional fault has been detected in realtime South America

13th Oct 01:12 PM

An additional fault has been detected in realtime Asia Singapore

13th Oct 01:15 PM

An additional fault has been detected in realtime Europe Frankfurt

13th Oct 01:17 PM

An additional fault has been detected in realtime Europe Ireland

13th Oct 01:59 PM

We are investigating the issue due to a faulty deployment.

13th Oct 02:39 PM

We have resolved the issue by forcibly removing all state from the cluster, and restarting the cluster. However, we are not sure of the root cause as yet, so will be focusing all of our energy on working out what went wrong today, and how we can prevent this in future. We'll update this ticket once we have a clearer idea of the problem.

Resolved

about 3 years ago