A streaming Dataflow pipeline was auto-optimized into a single merged step and is now exhibiting processing delays. How do you locate the pipeline bottleneck?
Choose an answer
Tap an option to check your answer.
Correct answer: Insert a Reshuffle operation after each processing step, and monitor the execution details in the Dataflow console..
Why this is the answer
The correct answer is to insert a Reshuffle operation after each processing step and monitor the Dataflow console. A Reshuffle operation prevents Dataflow from merging steps, forcing a materialization point. This allows you to isolate and observe the performance of individual processing stages in the Dataflow monitoring UI, making it easier to identify which specific step is causing the bottleneck. Inserting output sinks after each key processing step would add unnecessary I/O overhead and might not accurately reflect the in-memory processing bottlenecks. Logging debug information in each ParDo function can be verbose and difficult to analyze efficiently for performance issues, especially in a streaming pipeline. Verifying service account permissions is a good practice for general pipeline health but does not help in locating a processing bottleneck within an already running pipeline.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed