A Lambda function reads records from a Kinesis data stream and forwards them to a legacy REST API. About 10% of records have data errors and require manual handling. The Lambda has an SQS dead-letter queue configured for on-failure destinations, processes records in batches, and has retry logic. During testing many non-errored, already-processed records appear in the DLQ. Which event source option change will reduce the number of error-free records ending up in the DLQ?

Enabling splitting the batch when an error occurs (also known as BisectBatchOnFunctionError) is the correct solution. When a batch of records is processed by a Lambda function and an error occurs for even one record, the entire batch is typically retried. If this error is persistent (e.g., a data error), the whole batch will fail repeatedly and eventually end up in the DLQ, even if most records in the batch were processed successfully. By enabling batch splitting, Lambda automatically divides the failed batch into smaller sub-batches and retries them. This process continues until the problematic record is isolated, allowing successfully processed records to be acknowledged and preventing them from being sent to the DLQ. Increasing retry attempts would only delay the inevitable for the entire batch, still sending good records to the DLQ. Increasing concurrent batches per shard or decreasing the maximum age of a record would not address the core problem of good records being included in failed batches.