You are designing a data ingestion pipeline that must accommodate a variety of data formats including Avro, Parquet, and JSON. The pipeline must also perform transformations such as filtering and aggregation. Which GCP service should you use?
Choose an answer
Tap an option to check your answer.
Correct answer: Cloud Dataflow.
Why this is the answer
Cloud Dataflow is the correct choice because it is a fully managed service for executing Apache Beam pipelines, which are ideal for large-scale data processing, including ETL (Extract, Transform, Load) operations. It natively supports various data formats like Avro, Parquet, and JSON, and excels at complex transformations such as filtering and aggregation. Cloud Dataprep is for visual, code-free data preparation, not general-purpose pipeline orchestration. Cloud Pub/Sub is a messaging service for real-time data ingestion, not for complex transformations. BigQuery is a data warehouse for analytics, not for building data ingestion pipelines with transformations. Cloud Dataproc is a managed Apache Hadoop and Spark service, which can perform these tasks, but Dataflow is generally preferred for its fully managed, autoscaling nature and Apache Beam integration for stream and batch processing.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed