Amazon MLS-C01: Modeling — Supervised & Unsupervised (Classical ML) — Study Guide
Part of the AWS Machine Learning Specialty MLS-C01 — Study Guide. Practice with verified answers in the Amazon exam hub, or take timed practice tests on ExamRoll.io.
Model choice & architecture for supervised problems
Selecting a model begins with the data shape and production constraints. For linear signal with many examples and interpretable coefficients, regularized linear or logistic regression (SageMaker LinearLearner or scikit-learn ElasticNet) is fast and yields coefficients you can inspect. When nonlinear interactions dominate, tree ensembles such as XGBoost (SageMaker XGBoost container) or LightGBM capture heterogeneity without heavy feature scaling; tune eta (learning_rate), max_depth, subsample, colsample_bytree, and regularization alpha/lambda to control overfitting. SVMs can be effective on small-to-medium sized, well-scaled feature sets but typically require custom containers or scikit-learn ScriptMode on SageMaker because large datasets are expensive; remember to standardize features and choose kernel/regularization (C, gamma) carefully. Naive Bayes is a fast baseline for high-dimensional sparse categorical/text inputs; it assumes conditional independence and breaks when correlations are strong. For thousands of features or very large datasets, use dimensionality reduction (PCA) or feature hashing and prefer distributed training on ml.c5 or ml.p3 instances if using GPU-based deep nets. Deploy with SageMaker endpoints for real-time inference or Batch Transform for bulk predictions; use Multi-Model Endpoints when maintaining many models to reduce cost.
Metrics, imbalance, calibration, and deployment trade-offs
Accuracy is seductive but misleading on imbalanced problems; prefer class-sensitive metrics such as precision, recall, F1, ROC AUC for balanced-class emphasis, and PR AUC when the positive class is rare. When producing probabilities, check calibration with reliability diagrams and Brier score; use Platt scaling or isotonic calibration implemented offline (scikit-learn CalibratedClassifierCV) or calibrate within SageMaker training by outputting raw scores and applying a post-processing step. To handle class imbalance, prefer sample weighting (supported in SageMaker LinearLearner and many training scripts), targeted oversampling (SMOTE) or undersampling during training, and loss re-weighting or focal loss for neural nets. For near-real-time scoring of fraud probabilities, optimize model latency by using an instance type with low inference latency (ml.m5.large or CPU-optimized instances), keep models compact (prune or use distilled models), and use multi-model endpoints or A/B traffic shifting via SageMaker endpoints to rollout updates safely. Monitor production drift and data-quality with SageMaker Model Monitor and automate metrics logging to CloudWatch.
Feature engineering, multicollinearity and regularization
Multicollinearity inflates variance of coefficient estimates in linear models; detect it with variance inflation factor (VIF) and pairwise Pearson correlation, and mitigate by removing redundant features or using dimensionality reduction (PCA, SVD). Regularization is a principled remedy: L2 (Ridge) shrinks coefficients, L1 (Lasso) induces sparsity for feature selection, and ElasticNet combines both—these are available in scikit-learn and SageMaker LinearLearner. For strongly skewed numeric features apply log or Box–Cox transforms to stabilize variance and improve linear-model fits; remember that tree ensembles are robust to monotonic transforms while SVMs and neural nets need standardized inputs (StandardScaler) for stable training. High-cardinality categorical features benefit from target encoding, embeddings, or hashing; when target encoding is used, avoid leakage by computing encodings within cross-validation folds or using a separate holdout. Use SageMaker Feature Store to centralize and serve consistent feature transformations both to training and inference, and persist preprocessing code with model packages or Docker images so production transforms match training.
Unsupervised methods, clustering, anomaly detection, and interpretability
Clustering and anomaly detection are exploratory tools and preprocessing steps for supervised modeling. K-means (SageMaker k-means) is fast but assumes spherical clusters and requires scaled features; choose k with silhouette scores or elbow plots cautiously because inertia can be ambiguous. Density-based methods (DBSCAN) and hierarchical clustering capture non-spherical structure but are more computationally expensive. For anomaly detection at scale, consider SageMaker Random Cut Forest for streaming sensor data and Kinesis/Lambda pipelines for near-real-time scoring; tune tree count and sample size to control sensitivity. Interpretability tools are critical: use model-native feature importance for tree models and SHAP values (SageMaker Clarify or SHAP package) for local explanations and global effect summaries. Beware common traps: temporal leakage when random-splitting time-ordered data, over-relying on accuracy in imbalanced settings, and assuming independence for Naive Bayes. For deployment, package custom algorithms in Docker when needed, expose prediction and health endpoints, and orchestrate canary rollouts through endpoint traffic-splitting.
Practical Problem: FleetSight Logistics — Exploratory ML on Truck Images
Scenario: FleetSight collects ~100 GB/day of truck images stored in S3 and uses SageMaker Studio for experiments and SageMaker Ground Truth for image labeling. They need lightweight ML capabilities for defect detection and later intent to scale.
Challenge: Identify feasible supervised/unsupervised use cases and a low-cost pipeline for training initial models with limited labels and near-real-time inference for alerts.
Recommended Approach:
- Use SageMaker Ground Truth with active learning to label a seed set; store labeled data in S3 and register features in SageMaker Feature Store.
- Start with unsupervised anomaly detection using SageMaker Random Cut Forest on image-derived sensor metadata and simple image embeddings (extract embeddings with a pre-trained CNN in SageMaker Notebook using a GPU ml.p3.2xlarge).
- Train a lightweight supervised classifier using transfer learning (SageMaker built-in TensorFlow or PyTorch container) on the labeled subset; for quick baselines, extract embeddings and train an XGBoost model (SageMaker XGBoost) on CPU instances.
- Deploy the XGBoost model to a SageMaker endpoint with async inference or a Multi-Model Endpoint for cost-efficiency; monitor inputs and predictions with SageMaker Model Monitor and iterate labeling via Ground Truth.
Rationale: Use unsupervised embedding + Random Cut Forest to find candidate anomalies without many labels, then bootstrap supervised models via transfer learning and XGBoost for compact, fast inference; SageMaker services provide an integrated, scalable development and monitoring workflow.
← Exploratory Data Analysis · All domains · Deep Learning →
Practice these questions → · Timed practice on ExamRoll.io →
Pass the whole exam — not just this question
You found this answer. Get every verified question and explanation in one place, and save hours of prep. Free to start.
Pass your exam →