A company has three subsidiaries using different warehousing engines (Amazon Redshift, Teradata Vantage on AWS, and Google BigQuery). They want to consolidate data into an S3 data lake using Apache Iceberg. The new pipeline must connect to each source, run transformations using each source engine, join the results, and write to Iceberg with minimal operational effort. Which approach meets these requirements with the least operational overhead?
Choose an answer
Tap an option to check your answer.
Correct answer: Use the Amazon Athena federated query connectors for Amazon Redshift, Teradata, and BigQuery to build the pipeline in Athena. Write a SQL query to read from all the data sources, join the data, and run a Merge operation on the data lake Iceberg table..
Why this is the answer
The correct answer leverages Amazon Athena's federated query capabilities, which allow it to connect directly to various data sources (Redshift, Teradata via JDBC, BigQuery via connector) and execute SQL queries across them. This approach minimizes operational overhead because Athena is a serverless service, eliminating infrastructure management. It also allows for data transformation and joining using standard SQL, simplifying development. Finally, Athena can directly interact with Iceberg tables for write operations like MERGE. The other options involve more operational overhead or complexity: AWS Glue requires developing and managing ETL jobs. Amazon EMR necessitates managing clusters and writing PySpark code. Amazon AppFlow is primarily for SaaS application integration and would require additional services for complex transformations and Iceberg MERGE operations.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed