A security vendor receives telemetry every 30 seconds from thousands of global endpoints. Records are ~1 KB with 50 fields. Data is ingested into Kinesis Data Streams, and hourly summaries must be produced for Athena queries that will read 7–12 fields. Which approach requires the least customization to transform and store the ingested data?

Kinesis Data Analytics (KDA) is ideal for this scenario because it provides real-time processing of streaming data using SQL or Apache Flink, allowing for direct hourly aggregation and transformation with minimal custom code. It can then seamlessly send the processed data to Kinesis Data Firehose for delivery to S3. The other options involve more customization or less efficient workflows: Using Lambda to read and aggregate hourly would require managing the aggregation logic and state within Lambda, which is less efficient for continuous streams than KDA. Kinesis Data Firehose with a short-lived EMR cluster introduces significant operational overhead and complexity for hourly aggregations that KDA can handle natively. Kinesis Data Firehose with Lambda transformations is possible, but KDA offers a more integrated and often simpler solution for continuous aggregations directly on the stream.