Design a daily ETL workflow with these requirements: start when new data is uploaded to S3; once all needed uploads are present, run an ETL job that joins the new data with multiple-terabyte datasets in S3; write the join results back to S3; and notify an administrator if any job fails. Which architecture meets these requirements?
Choose an answer
Tap an option to check your answer.
Correct answer: Use an S3 upload event to trigger AWS Lambda, which starts an AWS Step Functions workflow that waits for all uploads to complete; call AWS Glue to perform the join; use Amazon CloudWatch alarms to send an SNS notification on failures..
Why this is the answer
This architecture effectively addresses all requirements. An S3 upload event triggering Lambda is a standard way to initiate workflows on new data. AWS Step Functions is ideal for orchestrating complex, multi-step workflows, including waiting for multiple S3 uploads to complete before proceeding. AWS Glue is a fully managed, serverless ETL service well-suited for joining large datasets in S3. CloudWatch alarms integrated with SNS provide robust failure notification. Incorrect options: SageMaker notebooks are interactive environments, not designed for automated, scheduled ETL workflows. AWS Batch can run jobs, but it doesn't inherently provide the orchestration needed to wait for multiple S3 uploads before starting the join. Chaining Lambda functions for direct joins on multi-terabyte datasets is inefficient and prone to Lambda's execution limits (memory, time).
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed