Design a data pipeline that auto-scales, delivers at-least-once processing, and preserves message ordering within 1-hour windows. Which architecture should you use?
Choose an answer
Tap an option to check your answer.
Correct answer: Use Cloud Pub/Sub for message ingestion and Cloud Dataflow for streaming analysis..
Why this is the answer
The correct architecture is Cloud Pub/Sub for message ingestion and Cloud Dataflow for streaming analysis. Cloud Pub/Sub is a fully managed, auto-scaling messaging service that provides at-least-once delivery and can preserve message ordering within a 1-hour window using ordered keys. Cloud Dataflow is a fully managed, auto-scaling service for executing Apache Beam pipelines, which naturally supports at-least-once processing and can handle large-scale streaming data efficiently. Apache Kafka is a strong alternative for message ingestion, offering similar guarantees, but it requires more operational overhead as it's not fully managed. Cloud Dataproc, while powerful for big data processing, is primarily designed for batch processing or long-running streaming jobs on managed clusters, making it less ideal for the dynamic auto-scaling and serverless nature implied by the problem for streaming analysis compared to Cloud Dataflow.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed