A partner drops a data file into an Amazon S3 bucket every day. An AWS Glue ETL job cleans and transforms the file daily and writes the result to Daily.csv in another S3 bucket. Sometimes the incoming file is empty or has missing required fields; when that happens, the company wants to keep the previous day's CSV. The engineer must ensure the prior day's file is overwritten only when the new daily file is complete and valid. Which solution accomplishes this with the least effort?

AWS Glue Data Quality (DQ) is designed precisely for this scenario. By enabling DQ within the ETL job and defining DQDL rules, the system can automatically validate the incoming data for completeness and validity (e.g., checking for empty files or missing required fields). If the data fails the DQ checks, the ETL job can be configured to abort or skip writing the output, thus preserving the previous day's valid file. This is the least effort because it's an integrated feature of Glue ETL. Triggering a Lambda function adds complexity and requires custom code for validation logic that Glue DQ can handle natively. Modifying the Glue ETL job to impute missing fields doesn't address the core requirement of not overwriting the previous file if the new data is invalid; it just attempts to fix bad data, which might not be desired. Running an Athena SQL query is a reactive approach that processes data after it's potentially written, and it doesn't inherently prevent overwriting the previous day's file if the new data is invalid.