Data collection downtime
Resolved
Jun 14 at 10:08pm BST
The incident has been resolved.
We will implement further systems to guard against this type of database misconfiguration, and are already working to improve the coverage of our monitoring systems to ensure this issue does not reoccur.
Affected services
Data collection service
Created
Jun 14 at 09:32pm BST
The data collection service was offline between approx. 18:10 and 20:50 (BST) on June 14th. This was caused by a misconfiguration in a critical database, and subsequent failures in our monitoring suite.
While data was not being collected, the status quo was maintained by Micromanager installs for the first 90 minutes, they then automatically reverted to their fallback functionality when the incident was recognised.
Service was restored by 21:20, but we are unable to backfill the missing data that was not collected during this time period.
Affected services
Data collection service