Load testing reveals a previously dormant Cloud Storage bucket servicing ~2000 writes/s and ~7500 reads/s; the Cloud Storage JSON API returns intermittent 5xx and 429 as demand spikes. How do you reduce API failures?

The correct answer is to limit the upload rate from your application clients. Cloud Storage buckets have initial I/O limits, and a sudden, massive increase in requests to a previously dormant bucket can exceed these limits, leading to 5xx and 429 errors. Gradually increasing the request rate allows Cloud Storage to scale up resources for the bucket, accommodating higher throughput without errors. Distributing uploads across many buckets is generally not necessary for performance and can complicate management; Cloud Storage is designed for high throughput on single buckets. Switching to the XML API offers no inherent performance advantage over the JSON API. Passing HTTP response codes back to clients doesn't solve the underlying issue of exceeding bucket limits.