A data engineer must orchestrate a set of ETL tasks in Python that include running Spark jobs on Amazon EMR, calling Salesforce APIs, and loading data into Amazon Redshift. The orchestration should handle retries and failures automatically. Which AWS service should the engineer use?
Choose an answer
Tap an option to check your answer.
Correct answer: Amazon Managed Workflows for Apache Airflow (Amazon MWAA).
Why this is the answer
Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is the correct choice because Apache Airflow is an open-source platform specifically designed for programmatically authoring, scheduling, and monitoring workflows. Its Python-based Directed Acyclic Graphs (DAGs) allow for complex orchestration, including Spark jobs on EMR, API calls, and data loading into Redshift, with built-in features for retries and failure handling. AWS Step Functions is suitable for state machine-based workflows but less ideal for complex, code-driven ETL pipelines like this. AWS Glue is an ETL service, not a workflow orchestrator for diverse tasks. Amazon EventBridge is an event bus for routing events, not for orchestrating multi-step data pipelines.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed