A data engineer set up an AWS Glue Data Catalog for objects in S3 and needs incremental updates. S3 event notifications are sent to an SQS queue. Which combination of steps provides incremental catalog updates with the least operational overhead? (Choose two.)
Choose an answer
Tap an option to check your answer.
Correct answer: Create an S3 event-based AWS Glue crawler that reads events from the SQS queue., Use an AWS Lambda function to update the Data Catalog directly based on S3 events received by the SQS queue..
Why this is the answer
The two correct options offer efficient, low-overhead methods for incremental Data Catalog updates. An S3 event-based AWS Glue crawler can directly consume SQS messages, automatically detecting and cataloging new or changed objects without requiring custom code or complex orchestration. This leverages Glue's built-in capabilities for schema inference and catalog management. Alternatively, an AWS Lambda function can be triggered by SQS messages. Within Lambda, you can use the AWS SDK to call Glue Data Catalog APIs (e.g., createtable, updatetable) to precisely manage catalog entries. This provides more granular control but requires writing and maintaining Lambda code. Scheduling a crawler is less efficient for incremental updates as it processes the entire S3 path each time, not just changes. Manually triggering is not automated and has high operational overhead. Using AWS Step Functions adds unnecessary complexity and overhead for this specific task, as Lambda or the event-based crawler can handle it more directly.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed