Amazon AIF-C01: MLOps & Deployment — Study Guide

By Laurynas Brazys · 30 July 2026 · #amazon #aif-c01 #certification #exam-prep #study-guide

Part of the AWS AI Practitioner AIF-C01 — Study Guide. Practice with verified answers in the Amazon exam hub, or take timed practice tests on ExamRoll.io.

Deployment and inference patterns: real-time, batch, serverless, and edge

Choosing an inference pattern begins with the service-level tradeoffs of latency, throughput, cost, and operational complexity. For low-latency, high-throughput needs, provisioned Amazon SageMaker real-time endpoints (single-model or multi-model endpoints) or SageMaker Real-Time Inference with autoscaling and GPU instances are typical; use endpoint variants to implement canary or blue/green rollouts. For large offline jobs where latency is not critical, SageMaker Batch Transform or distributed processing on AWS Glue/EMR is cost-effective and simple to scale. For unpredictable or low-QPS workloads, SageMaker Serverless Inference or invoking models from AWS Lambda (with provisioned concurrency when needed) reduces management overhead and cost. Edge deployments require packaging models with SageMaker Edge Manager and distributing them via AWS IoT Greengrass for devices with intermittent connectivity and strict latency or privacy needs. Common practitioner traps include choosing real-time endpoints for sparse traffic (leading to high cost), failing to test cold starts for serverless deployments, and underestimating memory/GPU needs for multi-model endpoints. Decisions should be guided by SLOs (p99 latency, throughput), model size, concurrency patterns, and expected frequency of updates; profiling with representative traffic is critical before settling on a hosting pattern.

Model registry, pipelines, and CI/CD for repeatable delivery

A robust production workflow uses SageMaker Model Registry to version artifacts, capture model lineage, and manage approval workflows, and SageMaker Pipelines to compose repeatable training, validation, and deployment steps. Integrating Git-based source control with AWS CodePipeline and CodeBuild enables model code CI: unit tests, data schema checks, and automated model evaluation that gates registry promotion. Deployment automation should implement staged environments (dev → staging → prod) and support automatic rollbacks or traffic shifting via endpoint variants, blue/green deployments, or Lambda-based routing. Feature stores (SageMaker Feature Store) ensure training and inference consistency by storing feature definitions and timestamps. Common traps include failing to record hyperparameters and environment versions, not automating validation tests (data schema, performance thresholds), and lacking immutable artifacts (store model artifacts in S3 or ECR). Use infrastructure-as-code (CloudFormation or CDK) for endpoint configuration, ensure reproducibility by pinning libraries and seeds, and plan for retraining triggers (data drift, periodic schedules) integrated with the same pipeline.

Monitoring, Model Monitor, and operational controls

Operational monitoring must cover data drift, concept drift, performance degradation, latency, and resource utilization. Amazon SageMaker Model Monitor can baseline training distributions and continuously profile inference traffic for feature drift, missing values, and schema changes; when ground truth is available, Model Monitor can also track prediction quality and target drift. Instrument inference logging to Amazon S3 and CloudWatch, and stream logs into Amazon Kinesis or Amazon EventBridge for real‑time alerting and automated pipeline triggers. Avoid common mistakes such as treating every drift alert as a retraining signal—transient shifts and seasonality can cause false positives—or setting thresholds without understanding natural variance. For imbalanced classes, prefer precision, recall, and F1 over accuracy to detect meaningful degradation. Implement RBAC and encryption (KMS) for model artifacts, use VPC endpoints and AWS PrivateLink for secure service access, and capture audit trails via AWS CloudTrail. Maintain operational playbooks for remediation: validate data pipelines, compare recent cohort performance, and either retrain, roll back, or apply calibration and feature fixes.

• Key evaluation metrics and when to use them
  • Accuracy: overall correctness; misleading on imbalanced data
  • Precision: correctness of positive predictions; use when false positives are costly
  • Recall (Sensitivity): ability to find positive cases; use when false negatives are costly
  • F1 score: harmonic mean of precision and recall; balanced choice for imbalanced classes
  • AUC-ROC: ranking quality across thresholds; useful for binary classifiers independent of threshold

Explainability, Model Cards, and compliance-ready controls

Explainability must be operational, auditable, and tuned to stakeholder needs. Amazon SageMaker Clarify provides pre-training bias checks, post-training bias metrics, and per‑prediction feature attributions via SHAP that can be produced at inference time or in offline batches. Runtime explanations should be logged alongside predictions and input fingerprints to provide traceability; store explanations in S3 and index them for retrieval. Model Cards document intended use, data provenance, evaluation metrics on relevant slices, known limitations, and performance caveats; maintain model card versions within the Model Registry to meet governance. Financial and regulated use cases require deterministic audit trails: preserve model artifacts, hyperparameters, evaluation data, and explanation logs; implement human-in-the-loop approval gates for high‑impact decisions and maintain counterfactual explanations for challenging cases. For secure integration with managed foundation models, use AWS PrivateLink interface endpoints to call Amazon Bedrock from within a VPC, encrypt payloads with KMS, and apply fine-grained IAM and network controls. Service choices for embedding storage and vector search depend on retrieval needs; compare options by performance and operational cost:

• Amazon OpenSearch Service (k-NN): scalable vector search with integrated analytics
• Amazon Kendra: managed semantic search with built-in connectors and enterprise features
• Amazon RDS (Postgres + pgvector): transactional vector storage, simpler for relational datasets
• Custom ANN on Amazon EKS or ECS: highest flexibility for NMSLIB/FAISS, but greater ops burden

Practical Problem: Use-Case Scenario

Scenario: FinSight operates a credit‑decisioning pipeline in AWS using SageMaker for model training, Bedrock for auxiliary generative explanations, and Amazon S3 for data. Models are deployed as SageMaker real-time endpoints inside a VPC, and Model Monitor is enabled to profile incoming features.

Challenge: Model Monitor has flagged feature drift beyond thresholds and business regulations require auditable, explainable decisions for any automated credit-limit changes.

Recommended Approach:

1. Use SageMaker Model Monitor alerts to trigger an EventBridge rule that sends the drift data to a SageMaker Pipelines retraining workflow, and snapshot current inference and recent input data to an S3 quarantine bucket.
2. Run an automated validation job within SageMaker Pipelines that compares recent data distributions to baseline, computes evaluation metrics (precision/recall/F1), and produces per-sample SHAP explanations with SageMaker Clarify for a representative cohort.
3. If automated checks pass, register the new model in SageMaker Model Registry with an updated Model Card capturing performance slices and explanations; use endpoint variant and traffic shifting for staged rollout. If checks fail, create an incident in the ops queue and invoke a human review step before any rollout.
4. For secure Bedrock calls that generate human-readable explanations, route Bedrock traffic via AWS PrivateLink and encrypt payloads with KMS; log all explanation outputs in S3 and link them to inference records for auditability.

Rationale: This approach combines automated detection, reproducible validation, and guarded deployment with audit trails and per-decision explanations—aligning operational controls with regulatory transparency and minimizing false retrain/rollback actions.

← Model Training · All domains · Cost Optimization →

Practice these questions → · Timed practice on ExamRoll.io →

Browse Amazon →