Amazon MLS-C01: Deployment, Inference & Serving (ML Implementation & Operations) — Study Guide
Part of the AWS Machine Learning Specialty MLS-C01 — Study Guide. Practice with verified answers in the Amazon exam hub, or take timed practice tests on ExamRoll.io.
Deployment patterns and serving options
Model serving choices balance latency, cost, throughput and operational complexity. Real-time SageMaker endpoints (provisioned or serverless) provide sub-100 ms to second-class latencies suitable for interactive APIs; choose instance families such as ml.c5/m5 for CPU models, ml.g4dn or ml.p3 for GPU models, or ml.inf1 for low-cost high-throughput Inferentia inference. Asynchronous inference and Batch Transform suit large-batch or variable-latency use cases: Batch Transform is ideal for large offline jobs (split big S3 objects into shards and use ml.m5/c5 or GPU instances as needed), while SageMaker Asynchronous Inference supports larger payloads with queuing for near-real-time batch. Multi-model endpoints host many models behind a single container, reducing storage overhead by loading models on demand; they work best when models are small, have modest cold-start costs, and access patterns are sparse. Choose SageMaker Serverless Inference for unpredictable, low-throughput workloads to avoid instance management; beware of cold starts and limited runtime. Key decision criteria include latency SLOs, per-invocation cost, concurrency pattern, model size, and cold-start tolerance. A common trap is using real-time endpoints for extremely high-volume batch workloads—this is expensive; instead use Batch Transform, Async Inference, or invoke endpoints via batching and concurrent workers.
Scaling, traffic shaping, and cost optimization
Autoscaling, traffic shifting and cost control must be co-designed with deployment topology. Use Application Auto Scaling to scale SageMaker endpoint variants with target-tracking policies based on invocation metrics or custom CloudWatch metrics; define sensible min/max capacities and cooldowns to avoid thrashing. For blue/green deployments and canary rollouts, use multi-variant endpoints or EndpointConfig updates with traffic-splitting percentages and gradual increases; integrate with AWS CodeDeploy to automate traffic shifting. Cost optimizations include model pruning, quantization (FP16 or int8), compiling with SageMaker Neo or AWS Neuron for Inf1, and moving nonlatency-sensitive workloads to Batch Transform or serverless inference. Spot instances reduce training costs but are not applicable for real-time endpoints; instead right-size instance types (cpu vs gpu vs inferentia) and consolidate models with multi-model endpoints where appropriate. Watch for traps like over-provisioning when using target-tracking without understanding per-instance throughput, or assuming multi-model endpoints eliminate memory limits—model loading still requires memory and may increase latency. Also compress model artifacts (ONNX, TF SavedModel with gz) and use lazy-loading strategies to cut storage and cold-start times.
Edge, low-latency inference, and pre/post-processing placement
For ultra-low-latency or disconnected environments deploy models to AWS IoT Greengrass (v2) or use SageMaker Edge Manager to package and monitor models on edge devices; compile with SageMaker Neo or AWS IoT Greengrass components and use quantization to meet memory/CPU constraints. Place preprocessing and feature extraction where it minimizes end-to-end latency and cost: simple filters can run in device firmware or Greengrass Lambda, batching and heavy transforms belong on an edge gateway or the cloud. For streaming event windows (e.g., sliding 10-minute windows), ingest with Amazon Kinesis Data Streams or Amazon MSK, use Kinesis Data Analytics or Flink/Apache Spark for windowing and aggregation, and forward summarized features to a SageMaker endpoint or a lightweight on-edge model—this reduces network egress and model invocation frequency. Beware of traps such as ignoring model versioning on edge devices, or failing to provision sufficient device storage for model artifacts. For server-side microservices, colocate preprocessing (API Gateway + Lambda or ALB + Fargate) to avoid cold-call overhead and reduce payload sizes sent to the model.
Monitoring, auditing, governance, and data handling
Operational ML requires continuous model health and data governance. Use SageMaker Model Monitor to baseline training data with a DataQualityJob and configure continuous monitoring to detect data drift, model quality regressions, missing values, and custom constraints; enable DataCaptureConfig on endpoints to capture inputs/outputs to S3 and trigger Model Monitor jobs. For feature-level lineage and auditing, use Amazon SageMaker Feature Store (online and offline stores) combined with AWS Glue Data Catalog and CloudTrail to track dataset access and transformations; Amazon Macie and Glue/SageMaker Processing jobs can discover and redact PII before training. Encryption pitfalls include SSE-KMS: when S3 objects are encrypted with a customer-managed CMK, ensure the SageMaker execution role has kms:Decrypt and kms:GenerateDataKey permissions and the CMK policy grants access; also ensure S3 bucket policies and VPC endpoints don’t block access. For large daily S3 objects (e.g., 100 GB), avoid single-file ingestion—partition into many smaller objects, store in Parquet and compress, and use Athena/Glue for schema detection. Common traps include insufficient monitoring schedules, not generating a proper baseline for Model Monitor, and forgetting to grant KMS access to all service principals (SageMaker, Glue, Lambda) that need decryption.
Practical Problem: Use-Case Scenario
Scenario: Streamlytic Media operates a podcast analytics platform on AWS. They use Kinesis Data Streams to ingest user events, store aggregated features in S3, and host models in SageMaker for real-time engagement prediction. Data contains occasional PII and is encrypted with an SSE-KMS customer-managed key.
Challenge: Provide low-latency predictions on a rolling 10-minute event window, redact PII before model training, ensure SageMaker can read encrypted S3 data, and implement continuous monitoring for feature drift.
Recommended Approach:
- Create a Kinesis Data Stream to ingest events, run Kinesis Data Analytics (Flink) to maintain rolling 10-minute windows and emit aggregated features to S3 in Parquet with hourly partitions.
- Detect and redact PII using Amazon Macie for discovery and a SageMaker Processing job (or AWS Glue job) to apply deterministic redaction/tokenization; store results in a Feature Store offline store for training.
- Grant the SageMaker execution role kms:Decrypt and kms:GenerateDataKey on the CMK, add the role to the CMK key policy, and ensure S3 bucket policy or VPC endpoint allows access for SageMaker.
- Deploy the model as a SageMaker real-time endpoint on ml.inf1 instances, enable DataCaptureConfig, create a Model Monitor baseline from training data, and configure continuous monitoring with alerts to CloudWatch.
Rationale: Streaming aggregation with Kinesis + Flink minimizes event volume and latency; redaction at processing time preserves privacy and compliance; explicit KMS permissions prevent access failures; Inferentia-backed endpoints and Model Monitor balance low inference cost with operational observability.
← Training · All domains · Security →
Practice these questions → · Timed practice on ExamRoll.io →
Pass the whole exam — not just this question
You found this answer. Get every verified question and explanation in one place, and save hours of prep. Free to start.
Pass your exam →