After deploying a new ingestion pipeline version, daily stored data in BigQuery increased 50%, Pub/Sub volumes unchanged, and only some daily partitions doubled. How do you investigate and remediate the cause?

The correct answer outlines a systematic approach to identify the root cause of increased data. Checking for duplicate rows in affected partitions directly addresses the observed data increase. BigQuery Audit logs help pinpoint which jobs wrote the data, linking the issue to specific ingestion processes. Cloud Monitoring then allows correlating job execution and pipeline versions with the deployment, identifying the faulty pipeline. Finally, stopping older pipeline versions ensures only the intended, potentially corrected, version is writing, preventing further data duplication. Option 1 (incorrect) focuses on deduplication as a primary solution rather than root cause analysis and remediation. While deduplication might be necessary, it doesn't prevent recurrence. Sharing a script is a reactive measure. Option 2 (incorrect) suggests restoring BigQuery tables via time travel as a primary step, which is a drastic measure before fully understanding the problem. It also focuses on code errors and multiple writes without a clear path to identify the specific faulty pipeline version. Option 4 (incorrect) immediately rolls back and restores, which is disruptive and doesn't provide insight into the cause. Replaying messages without understanding the source of duplication could reintroduce the problem.