A company runs an AWS Glue ETL job every day at the same time to process XML data stored in an Amazon S3 bucket. New data is added to the S3 bucket daily. A solutions architect notices that AWS Glue is reprocessing all the data each run. What should the solutions architect do to prevent AWS Glue from reprocessing old data?
Choose an answer
Tap an option to check your answer.
Correct answer: Edit the job to use job bookmarks..
Why this is the answer
Using job bookmarks is the correct solution because they track previously processed data, allowing AWS Glue to process only new or changed data in subsequent runs. This prevents reprocessing of old data, saving time and resources. Deleting data after processing is not a viable solution as it would remove the source data, which might be needed for other purposes or future analysis. Setting NumberOfWorkers to 1 would reduce parallelism and slow down the job without addressing the reprocessing issue. A FindMatches ML transform is used for finding duplicate records within a dataset, which is unrelated to preventing reprocessing of already processed data.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed