Streaming IoT data from Cloud Pub/Sub through Dataflow to BigQuery shows ~2% corrupt records in a preview. How should you modify the Cloud Dataflow pipeline to filter out corrupt data?

A ParDo transform is the most suitable choice for filtering individual elements in a Dataflow pipeline. It allows you to apply custom logic to each record, identify corrupt ones, and then either discard them or route them to a separate error sink. This directly addresses the need to filter out corrupt data. SideInputs are for enriching data with external information, not for filtering individual elements based on their own content. A Partition transform divides a PCollection into multiple PCollections based on a key, which isn't designed for simple filtering of corrupt records. GroupByKey aggregates elements with the same key, which is not applicable here as the goal is to filter individual corrupt records, not group them.