Amazon MLA-C01: Model Deployment and Inference — Study Guide
Part of the AWS Machine Learning Engineer Associate MLA-C01 — Study Guide. Practice with verified answers in the Amazon exam hub, or take timed practice tests on ExamRoll.io.
Real-time, serverless, async, and batch endpoints — core concept
Real-time inference in Amazon SageMaker is a low-latency, stateful service model where you create a Model, an EndpointConfig and an Endpoint that allocates provisioned compute (ml.* instance types) and stays available to serve requests via the InvokeEndpoint API. CreateEndpointConfig accepts ProductionVariants, and each ProductionVariant defines ModelName, InitialInstanceCount, InstanceType, and InitialVariantWeight; you can change traffic and capacity with UpdateEndpointWeightsAndCapacities or UpdateEndpoint. For predictable low-latency needs, provisioned real-time endpoints are the primary option and support multi-container inference pipelines to chain preprocessing, model, and postprocessing containers.
Serverless Inference removes instance management and is configured at the endpoint level with a ServerlessConfig that specifies MemorySizeInMB and MaxConcurrency for each ProductionVariant; SageMaker manages container provisioning and scales to zero when idle, making it ideal for spiky, low-throughput workloads. Asynchronous Inference is optimized for requests that take a long time to run or where the client does not require synchronous response. An async endpoint is created with AsyncInferenceConfig in CreateEndpointConfig (OutputConfig with S3OutputPath, optional ClientConfig, and MaxConcurrentInvocationsPerInstance) and clients call InvokeEndpointAsync, providing an input S3 URI; results are written to the configured S3 output location. Batch Transform is a separate job type (CreateTransformJob) for large offline inference workloads; the API requires TransformInput (S3DataSource with S3Uri and S3DataType), TransformOutput (S3OutputPath, Accept, AssembleWith), and TransformResources (InstanceType, InstanceCount). Batch Transform is best when throughput but not latency matters and it supports large parallelism across datasets.
Multi-model endpoints, inference pipelines, shadowing and A/B testing — key services and configuration
When hosting many models with low per-model QPS, SageMaker Multi-Model Endpoints (MME) let a single container host dozens to thousands of model artifacts stored in S3 and load them on demand. You build a model server container that implements the SageMaker multi-model server pattern or use a supported framework image, upload model tarballs to S3, and create a Model resource that references the container. At invocation time you pass the target model name via the InvokeEndpoint API parameter TargetModel (or the header X-Amzn-SageMaker-Target-Model) so the server loads that model from S3 into memory. MMEs save memory and operational cost for large model fleets, but add cold-load latency for models not yet resident in the runtime.
Inference pipelines are implemented as multi-container models where the Model resource lists Containers in order; the endpoint routes payloads through the first container (preprocessing), then the model container, then the postprocessing container. Define each container with its own ModelDataUrl and environment variables in CreateModel. For canary or blue/green style testing, use multiple ProductionVariants in an EndpointConfig and control traffic split with InitialVariantWeight and later via UpdateEndpointWeightsAndCapacities. A shadow deployment can be achieved by either sending a copy of each request at the application layer to a shadow endpoint (no traffic weight on the production endpoint) or creating a low-weight ProductionVariant so the endpoint infrastructure receives some mirrored traffic; application-layer request duplication gives you full isolation of the experiment and independent observability.
For on-demand and continuous monitoring, configure DataCaptureConfig when creating an endpoint to persist request and response payloads to S3. DataCaptureConfig fields of interest include EnableCapture (true), InitialSamplingPercentage, DestinationS3Uri, and CaptureOptions (REQUEST, RESPONSE). Captured data becomes the basis for SageMaker Model Monitor and SageMaker Clarify post-deployment checks; you can create baselines with Model Monitor’s CreateMonitoringSchedule and run ad-hoc Processing jobs that use the built-in model-monitoring container to compute constraints and drift metrics.
Design patterns and trade-offs
Choose provisioned real-time endpoints when you need single-digit to low-double-digit millisecond latency and you can afford the always-on capacity. If cost-per-minute while idle is the dominant constraint and traffic is intermittent, serverless endpoints reduce operations: configure ServerlessConfig.MemorySizeInMB and ServerlessConfig.MaxConcurrency for each variant and let SageMaker autoscale. For workloads with long-running inferences or heavy payload exchange patterns, asynchronous endpoints decouple client lifetime from compute; they require S3 for inputs/outputs and are most appropriate when clients can poll or receive S3 notifications for completion.
Multi-model endpoints reduce memory duplication and S3 object management complexity, but they add per-model cold-start latency and require a model server capable of on-demand S3 loading and proper lifecycle management (eviction/LRU). If per-model latency is critical, host hot models on dedicated ProductionVariants and offload low-traffic models to an MME. Inference pipelines centralize preprocessing and postprocessing logic closer to the model, reducing client-side code and ensuring consistent transformation between training and inference, but they increase endpoint startup complexity and demand robust container contract design (input/output codecs and content types).
A/B testing using ProductionVariant weights is straightforward for traffic splitting and offline metrics collection, but when you want to shadow traffic without affecting production metrics, prefer application-level mirroring. For progressive rollouts and rollback automation, integrate UpdateEndpointWeightsAndCapacities into a CodePipeline or Step Functions workflow that includes automatic metric evaluation using CloudWatch metrics, Model Monitor alerts, and a manual approval action that gates final promotion.
Common pitfalls and decision criteria
A common operational mistake is assuming Model Monitor will detect label availability issues; Model Monitor can detect feature distribution drift and data quality violations from captured requests, but to measure label-based metric degradation (F1, recall) you must deliver ground-truth labels back to S3 in a format the monitoring jobs can consume and schedule a monitoring job that calculates prediction vs. truth. Another pitfall is failing to size ServerlessConfig.MemorySizeInMB appropriately; under-provisioned memory causes throttling or container crashes, while over-provisioning increases cost. For multi-model endpoints, neglecting to set appropriate S3 object layout and lifecycle (prefixes, model manifests) makes cold loads slower and complicates eviction policies.
When dealing with class imbalance for fraud detection, prefer algorithm-native weighting to heavy sampling pipelines for minimal operational overhead; for example, XGBoost (SageMaker XGBoost container) supports ‘scale_pos_weight’ hyperparameter which you compute as negative_examples/positive_examples and pass via the hyperparameters map in the CreateTrainingJob call. For manual deployment gating, use the SageMaker Model Registry: create a ModelPackageGroup, call CreateModelPackage to register a model package and set a model package status to PendingManualApproval; an external CodePipeline manual approval action or a Step Functions + SNS manual confirmation can then call UpdateModelPackage to set ApprovalStatus = “Approved” before CreateModel or CreateEndpoint are run.
Practical Problem: Use-Case Scenario
FraudDetectCo is building an online fraud detection system that must consolidate transaction logs in S3 and on‑prem MySQL customer profile tables, train an XGBoost model, deploy it with near-real-time latency, enforce a manual approval gate for production releases, and detect both dataset anomalies and model drift on demand.
Data aggregation and preprocessing: use AWS Database Migration Service (DMS) or AWS Glue’s JDBC connector to continuously replicate the on‑prem MySQL tables into S3 (parquet) or into an Amazon RDS/Athena-enabled data lake; catalog with AWS Glue and register features in Amazon SageMaker Feature Store offline store to provide consistent training and online-serving feature lookup. This centralizes feature lineage and enforces S3 security policies and Lake Formation governance for isolation.
Training and class imbalance handling: run SageMaker Training jobs using the SageMaker XGBoost built-in container. Compute the training label ratio and set the XGBoost hyperparameter “scale_pos_weight” in the CreateTrainingJob HyperParameters map to address class imbalance with minimal preprocessing. Use Pipe mode for the training channel (DataSource with S3DataSource and S3DataType set to S3Prefix, and enable “RecordWrapperType”:“None” if using Pipe mode) to reduce startup time and data download latency across consecutive jobs.
Model registry and manual approval: register trained models in the SageMaker Model Registry by calling CreateModelPackage within a ModelPackageGroup. Set the initial ApprovalStatus to PendingManualApproval, integrate an AWS CodePipeline that includes an AWS Manual Approval action, and upon manual confirmation call UpdateModelPackage with ApprovalStatus=“Approved” before promoting the ModelPackage to production via CreateModel and CreateEndpointConfig.
Deployment and inference topology: deploy the model to a provisioned real-time endpoint for low-latency scoring. If hundreds of models must be hosted later, evaluate a Multi-Model Endpoint and use the InvokeEndpoint TargetModel parameter to direct specific models stored in S3. Configure DataCaptureConfig (EnableCapture=true, InitialSamplingPercentage=100, DestinationS3Uri=s3://
<bucket>/captures, CaptureOptions=[‘REQUEST’,‘RESPONSE’]) to collect request/response payloads for on-demand analysis.Monitoring and anomaly detection: schedule SageMaker Model Monitor baselines with CreateMonitoringSchedule for data quality and feature drift. For dataset-level anomaly detection and visualization, feed the captured S3 data into Amazon Lookout for Metrics to automatically detect anomalies and into Amazon QuickSight for dashboards. For on-demand bias and drift assessment, run SageMaker Clarify processing jobs against the captured data or launch an ad-hoc Model Monitor Processing job (via CreateProcessingJob) that applies the stored baseline constraints and produces the comparison report.
Rationale: this approach centralizes features for reproducible training and low-latency serving, uses XGBoost’s scale_pos_weight for class imbalance with minimal pipeline complexity, enforces a manual approval in the Model Registry integrated with CodePipeline, reduces training startup latency by using Pipe mode for streaming training data, and provides both automated anomaly detection (Lookout for Metrics) and on-demand fairness/drift checks (Clarify + Model Monitor) using captured inference data.
← Model Evaluation and Selection · All domains · MLOps and Model Lifecycle Management →
Practice these questions → · Timed practice on ExamRoll.io →
Pass the whole exam — not just this question
You found this answer. Get every verified question and explanation in one place, and save hours of prep. Free to start.
Pass your exam →