All times are shown in UTC
Following a routine recycle of our databases, our realtime system is attempting to connect to old database servers in all regions that do not exist.
This has resulted in an increased rate of 500, 401 and timeout errors.
We are looking into this now.
We have identified the root cause of the issue and have fixed it.
We are also going to modify our monitoring systems to better detect intermittent failures as our monitoring systems were unfortunately reporting everything as healthy.
Resolvedin about 5 hours