An external partner drops daily CSVs with some malformed rows into GCS. You need to load into BigQuery and inspect bad rows. Which pipeline design is appropriate?
Choose an answer
Tap an option to check your answer.
Correct answer: Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis..
Why this is the answer
A Dataflow batch pipeline is the most appropriate solution because it allows for robust error handling and transformation of data before loading into BigQuery. It can process the CSVs, identify malformed rows, load valid data into the primary BigQuery table, and direct invalid rows (with associated error messages) to a separate dead-letter table for later inspection and analysis. Federated data sources allow querying external data but don't provide a mechanism to separate good from bad records during loading or to store bad records for analysis. BigQuery monitoring in Stackdriver is for operational metrics and alerts, not for data validation and error handling during ingestion. Importing with gcloud CLI and maxbadrecords=0 would cause the entire load job to fail if even one malformed record is found, preventing any data from being loaded and not providing a way to inspect the bad rows.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed