Back to overview
Downtime

Data collection downtime

Jun 14 at 09:32pm BST
Affected services
Data collection service

Resolved
Jun 14 at 10:08pm BST

The incident has been resolved.

We will implement further systems to guard against this type of database misconfiguration, and are already working to improve the coverage of our monitoring systems to ensure this issue does not reoccur.

Created
Jun 14 at 09:32pm BST

The data collection service was offline between approx. 18:10 and 20:50 (BST) on June 14th. This was caused by a misconfiguration in a critical database, and subsequent failures in our monitoring suite.

While data was not being collected, the status quo was maintained by Micromanager installs for the first 90 minutes, they then automatically reverted to their fallback functionality when the incident was recognised.

Service was restored by 21:20, but we are unable to backfill the missing data that was not collected during this time period.