A company uploads CSV files to an Amazon S3 bucket. An AWS Glue crawler builds tables and schemas for the data, and an AWS Glue job processes the tables and writes the results into an Amazon Redshift database. The Glue job handles column mapping and creates the Redshift tables as needed. Re-running the Glue job causes duplicate rows to appear in the Redshift tables. The company needs a way to update the Redshift tables without introducing duplicates. Which approach satisfies this requirement?

The correct approach involves using a staging table in Redshift. This allows the Glue job to first load new data into a temporary table. Then, SQL commands can be executed to perform an UPSERT operation (update existing rows and insert new ones) from the staging table into the main Redshift table. This ensures data integrity by preventing duplicates and updating records efficiently. Incorrect options: Loading data into a MySQL database introduces unnecessary complexity and latency, as the final destination is Redshift. dropDuplicates() in Spark DataFrames only removes duplicates within the current dataset being processed, not against existing data in the Redshift table. It wouldn't prevent duplicates if the same record already exists in Redshift from a previous run. ResolveChoice is an AWS Glue transform used for handling schema evolution in dynamic frames, specifically for resolving data type conflicts or choosing a specific value from multiple possibilities for a column. It does not address the problem of preventing duplicate rows when writing to a target database.