A legacy application was migrated into a Lambda function that performs many third-party API calls to pull data at month end, then processes it to produce monthly reports. The third-party provider introduced limits: a fixed number of API calls per minute and per day (provided in response headers), returning errors when limits are exceeded. The entire data-processing run may now take days if the app must throttle to avoid exceeding limits. What is the most operationally efficient way to refactor this serverless workflow to respect those limits?

Step Functions is ideal for orchestrating complex workflows with retries and delays. A state machine can be designed to make API calls, catch rate limit errors, and then use a Wait state to pause execution for a specified period before retrying. This allows for controlled throttling and ensures the application respects third-party limits without manual intervention, making it operationally efficient. Sending tasks to an SQS queue and polling with Lambda could work but adds complexity in managing the polling rate dynamically to stay within limits. CloudWatch Logs metrics and alarms are for monitoring and alerting, not for actively controlling application flow or implementing retry logic. Kinesis Data Firehose is for data ingestion and delivery, not for orchestrating API calls with rate limiting and retries.