RetailVision has 240 per-store personalization models (each ~120 MB) that are queried infrequently (minutes between requests per model). They currently host each model as a separate endpoint, which is costly. They want to consolidate to minimize cost while keeping acceptable cold-start latency when a model is first requested. What is the best SageMaker-hosting approach to reduce cost in this scenario?

The correct option leverages SageMaker multi-model endpoints, which are designed for cost-effective hosting of many models with infrequent access. By storing model artifacts in S3 and loading them into memory on demand, it consolidates numerous low-traffic models onto fewer instances, significantly reducing hosting costs. The trade-off is acceptable cold-start latency when a model is first requested, which aligns with the problem statement's tolerance. The incorrect options are less suitable: Replacing SageMaker with EC2 and loading all models into memory is expensive and inefficient, as most models are infrequently queried. Using 240 multi-container endpoints with autoscaling to zero is still costly due to the overhead of managing many individual endpoints and the cold-start issues associated with scaling to zero. SageMaker Asynchronous Inference is for long-running inferences and doesn't address the cost of 240 separate endpoints or the cold-start issue for real-time requests.