A company uploads data to an S3 bucket daily and uses the AWS Glue Data Catalog. Some days a daily report runs before all data arrives. The data engineer must send a message identifying any incomplete data to an existing Amazon SNS topic with minimal operational overhead. Which solution meets this need?
Choose an answer
Tap an option to check your answer.
Correct answer: Create data quality checks and use AWS Glue workflows to perform those checks. If a dataset is incomplete, have the workflow generate an EventBridge event and configure EventBridge to forward that event to the SNS topic..
Why this is the answer
The correct solution leverages AWS Glue workflows for data quality checks and Amazon EventBridge for notifications, offering minimal operational overhead. Glue workflows are ideal for orchestrating ETL processes, including data quality validation, within the Glue ecosystem. If a check fails, the workflow can emit an EventBridge event. EventBridge then acts as a serverless event bus, easily routing this event to an existing Amazon SNS topic, ensuring the engineer is notified without managing additional infrastructure. Incorrect options: AWS-managed Apache Airflow introduces more operational overhead and complexity than needed for this specific task, despite being a powerful orchestrator. Amazon EMR clusters are suitable for large-scale data processing but are overkill and more expensive for simple data completeness checks, incurring higher operational overhead. AWS Lambda functions orchestrated by Step Functions could work, but using Glue workflows for data quality checks directly within the Glue environment is often more integrated and simpler when already using the Glue Data Catalog, reducing the need for separate Lambda deployments for this specific task.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed