Amazon MLS-C01: Data Engineering & Feature Engineering — Study Guide
Part of the AWS Machine Learning Specialty MLS-C01 — Study Guide. Practice with verified answers in the Amazon exam hub, or take timed practice tests on ExamRoll.io.
Data ingestion, storage, and file formats
Designing an ML-ready data lake starts with picking the right ingestion pattern and persistent format. For real-time telemetry use Kinesis Data Streams (low-latency processing) or Kinesis Data Firehose (managed delivery). Firehose can deliver directly into Amazon S3 and perform server-side record format conversion to Parquet/ORC when combined with AWS Glue Schema Registry and a Lambda transform; choose Data Streams + Kinesis Data Analytics or Lambda if you need custom enrichment or sub-second responses. For bulk migration, use AWS DataSync, Database Migration Service (DMS) for RDS/OLTP sources, or Snowball for terabyte-scale offline transfers. On S3, store columnar Parquet for analytic workloads (predicate pushdown, compression like Snappy, partition keys tuned to query patterns) and TFRecord when feeding TensorFlow pipelines for high-throughput sequential reads. Beware of small-file explosion: buffer writes (Firehose buffering, Glue batching) to produce S3 objects sized for big-data frameworks (tens to hundreds of MB). Schema evolution is common: use Glue Catalog and Schema Registry to version schemas; use partitioning and naming conventions to avoid expensive full-table scans. A common trap is using File mode in downstream processing that copies whole datasets to compute nodes; prefer streaming (SageMaker Pipe mode), Redshift Spectrum, or Athena over wholesale copying when datasets are millions of records.
Serverless and cluster ETL: Glue, EMR, Redshift, and SageMaker
ETL choices hinge on scale, customization, cost profile, and library needs. AWS Glue provides serverless Spark ETL with cataloging, crawlers, and built-in job orchestration—excellent for frequent, event-driven transforms where you want managed scaling. EMR is preferable when you need custom Spark/Hadoop ecosystems, long-running clusters, or specialized libraries (GraphX, MLlib custom builds) and you can use S3 as the underlying data lake. For analytics without ingestion, use Redshift Spectrum or Athena to query Parquet/ORC in S3 directly; Redshift is better for complex joins and BI workloads. For model-data preparation, SageMaker Processing jobs provide managed containers to run scalable Spark or Python preprocessing, ensuring preprocessing code runs close to training and can be versioned with the training job. When launching SageMaker training with built-in algorithms, supply at minimum the training image, IAM role, instance_type/count, and S3 URI for training data; consider Pipe mode to stream records and avoid EBS copying for multi-million record datasets. Common traps: choosing Glue for extremely low-latency transforms (use Lambda/Kinesis instead) or copying S3 data into HDFS/EBS unnecessarily when Redshift Spectrum/Athena would suffice.
Feature engineering, transformation pipelines, and selection
Feature engineering should be reproducible and isolated from leakage. Implement preprocessing as production-grade pipelines (scikit-learn Pipeline, Spark ML pipelines, or SageMaker Processing + Feature Store) so identical transformations apply at training and inference. Handle missingness with strategy selection: simple imputation (mean/median/mode) for small gaps; model-based methods (KNN, iterative MICE) when other features predict missing values. Scale numeric features with StandardScaler or MinMax for gradient-based models; apply robust scaling if outliers dominate. For categorical variables, choose one-hot encoding for low-cardinality, target encoding or learned embeddings for high-cardinality categories (use Embeddings in deep models or feature hashing for dimensionality control). For text, perform consistent normalization (lowercase, punctuation, tokenization); use Word2Vec/BlazingText for embeddings or TF-IDF/sparse pipelines for linear models; BlazingText in SageMaker supports skip-gram/CBoW and is efficient at scale. For feature selection, use L1 regularization, tree-based importance (XGBoost/RandomForest), mutual information, or recursive feature elimination, and always perform feature selection inside cross-validation folds to avoid selection bias. The biggest practical trap is leakage: never derive features using future-target information or apply preprocessing using the full dataset before splitting.
Security, access control, governance, and operational monitoring
Secure and auditable data engineering is mandatory. Encrypt data at rest using SSE-KMS for S3 and EBS volumes and use client-side encryption for additional control; manage keys with AWS KMS CMKs and use key policies and grants for least privilege. Restrict S3 datasets to a VPC using an S3 VPC endpoint and precise bucket policies or S3 Access Points scoped to the VPC principal; combine with IAM roles attached to SageMaker, Glue, EMR, or Lambda to enforce least privilege. For sensitive PII use tokenization or field-level encryption and ensure KMS rotates keys per policy. Operationally, enable CloudTrail for SageMaker and Glue API activity logging and CloudWatch for job metrics; use SageMaker Model Monitor for drift detection and set alerts to retrain or bias-check models. Data governance requires the Glue Data Catalog or an enterprise metadata store, data lineage, and tagging for lifecycle policies; implement S3 lifecycle rules (glacier transition) for cold data. Common misconceptions include assuming security groups alone are sufficient (you also need IAM, bucket policies, and VPC endpoints), or forgetting to enable encryption on training instance volumes—always include EBS encryption in training job definitions and validate access with least-privilege roles.
Practical Problem: Use-Case Scenario
Scenario: MetroRetail runs an AWS ML environment with customer clickstream ingested into Kinesis Data Streams, stored in S3, and models trained in SageMaker. The data lake uses Glue Catalog and Athena for analysts.
Challenge: Convert streaming CSV click events into Parquet in S3 with schema evolution, produce reliable feature vectors for a recommendation model, and ensure the pipeline scales without copying multi-gigabyte datasets to training instances.
Recommended Approach:
- Use Kinesis Data Streams for ingestion, attach a Lambda consumer to perform lightweight validation and forward records to a Kinesis Data Firehose delivery stream configured with Glue Schema Registry to convert JSON/CSV to Parquet and write partitioned Parquet files to S3 (buffer hints tuned to ~64–128 MB).
- Catalog Parquet in AWS Glue; use Glue ETL jobs (serverless Spark) or SageMaker Processing (Spark container) to build reproducible feature pipelines, applying imputation (median for skewed numerics), StandardScaler, and categorical embeddings for high-cardinality item_id.
- Store online feature vectors in SageMaker Feature Store (online store for low-latency lookups) and offline features in S3 (feature store offline store via Parquet). Train in SageMaker using Pipe mode pointing to S3 Parquet manifests to stream data and avoid full copies.
- Secure S3 with SSE-KMS, restrict access using an S3 VPC endpoint and tight bucket policies for the VPC, and enable CloudTrail + SageMaker Model Monitor for activity logging and drift alerts.
Rationale: This approach leverages managed streaming-to-Parquet conversion and serverless ETL to scale, uses Feature Store for consistency between training and inference, avoids costly data movement with Pipe mode, and enforces encryption and least-privilege access for production readiness.
All domains · Exploratory Data Analysis →
Practice these questions → · Timed practice on ExamRoll.io →
Pass the whole exam — not just this question
You found this answer. Get every verified question and explanation in one place, and save hours of prep. Free to start.
Pass your exam →