A company runs multiple AWS Glue ETL jobs that read S3 data using a DynamicFrame and load transformed data into Amazon RDS for MySQL once per day. Currently the jobs reprocess all objects in the S3 bucket, but the company wants the jobs to operate only on the daily incremental data. Which solution requires the LEAST amount of code changes?
Choose an answer
Tap an option to check your answer.
Correct answer: Enable job bookmarks for the ETL jobs to update the state after a run to keep track of previously processed data..
Why this is the answer
Enabling job bookmarks for AWS Glue ETL jobs is the most efficient solution because job bookmarks automatically track and prevent reprocessing of previously processed data from S3. This feature is designed specifically for incremental processing and requires minimal code changes, often just a configuration setting. Creating an ETL job to log S3 file status in DynamoDB would require significant custom code development and maintenance to manage state. Enabling job metrics in CloudWatch would provide monitoring but not prevent reprocessing; it doesn't manage the state of processed data. Configuring ETL jobs to delete processed objects from S3 is generally not recommended for data integrity and auditing purposes, and it would also require code changes to implement the deletion logic.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed