ID Verification Timeouts
Incident Report for Synapse
Postmortem

What happened: ID verification as returning ID's from 2:50pm to 3:30pm on Friday January 5th. The issue was a problem with one of our hosts, essentially the storage on the host got filled up.

How we fixed it: Replaced hosts

How we are doing to ensure it will not happen again: We are putting on every host a service that cleans up old container images. That should prevent it from running out of space again. We have also added more logging to be able to catch these type of issues faster in the future.

Who was affected: Due to this issue, our government ID verification stack was performing at a lower success rate (around 50%). So IDs verifications were partially degraded during this timeframe.

Posted Jan 09, 2018 - 15:33 PST

Resolved
This incident has been resolved.
Posted Jan 09, 2018 - 15:32 PST