A company is developing machine learning models on AWS as independent microservices. Each microservice fetches about 1 GB of model data from Amazon S3 at startup and loads it into memory. Users access the ML models via an asynchronous API and can send single or batch requests. The company serves hundreds of users. Usage patterns are irregular: some models may be idle for days or weeks while others receive batches of thousands of requests. Which solution will meet these requirements?

The correct solution uses SQS for asynchronous processing and ECS for flexible, scalable microservices. SQS decouples the API requests from the ML model processing, handling irregular usage patterns and batch requests efficiently. ECS allows for persistent model data in memory, avoiding repeated 1GB S3 fetches at startup, which Lambda would incur for each invocation. Auto scaling for ECS based on SQS queue size ensures resources are provisioned only when needed, optimizing cost and performance. Incorrect options: NLB with Lambda: Lambda functions have a 15-minute timeout, which might be insufficient for large batch ML inferences. Also, loading 1GB of data for every cold start is inefficient and costly. ALB with ECS: While ECS is suitable, direct invocation via ALB is synchronous. The requirement for an asynchronous API and irregular usage patterns makes SQS a better choice for decoupling. SQS with Lambda: Again, Lambda's cold start overhead for 1GB data and its timeout limits make it less ideal for this specific ML workload compared to ECS. Lambda does not scale vCPUs; it scales by concurrent invocations.