Increased errors on management api leading to project operation failures

Updates timeline

1. Investigating Oct 6, 07:53 AM UTC

   We are investigating an increase in errors on project update flows including creation, pausing/unpausing and upgrading across multiple regions.

2. Investigating Oct 6, 08:10 AM UTC

   We are continuing to investigate this issue.

3. Monitoring Oct 6, 09:11 AM UTC

   We have identified a rollout causing error increase on management api dependant services, which is causing project operation failures and FGA Authentication errors across multiple functions. We have implementing a fix and observing recovery. Our teams are continuing to monitor for full resolution.

4. Resolved Oct 6, 09:11 AM UTC

   This issue has been resolved and our error rates have returned back to normal. During this incident users would have observed unauthorized/5xx failures across multiple operations on the project operations and dashboard functions. We have since rolled out a fix and all functionality is restored.