You need a serverless, real-time ingestion and analytics pipeline for high-volume streaming JSON events. The pipeline must buffer data, convert JSON to a columnar format optimized for queries without losing data, store results in a highly available datastore, and let analysts run SQL queries and connect BI dashboards. Which design best meets these requirements?
Choose an answer
Tap an option to check your answer.
Correct answer: Define a schema in a managed Data Catalog for the incoming format. Use a managed streaming delivery service to ingest the events, transform them to a columnar format (Parquet or ORC) using the Data Catalog, and deliver results to object storage. Let analysts query the object store using a serverless SQL query engine and connect BI tools with its JDBC connector..
Why this is the answer
The correct option leverages a managed streaming delivery service (like Kinesis Data Firehose) for real-time ingestion and buffering, which can automatically convert JSON to columnar formats (Parquet/ORC) using a Data Catalog schema (like AWS Glue Data Catalog) for schema enforcement and optimization. Delivering to object storage (S3) provides a highly available and cost-effective datastore. A serverless SQL query engine (like Amazon Athena) directly queries the columnar data in S3, allowing analysts to use SQL and connect BI tools via JDBC, fulfilling all requirements. The other options have drawbacks: Writing individual JSON records to object storage and triggering a serverless function for each record is inefficient for high-volume streaming and can lead to many small files, impacting query performance. Inserting processed records into a managed relational database is not ideal for high-volume streaming data due to potential write bottlenecks and higher costs compared to object storage for analytics. A real-time SQL stream-processing service (like Kinesis Data Analytics) can convert to Parquet, but the initial ingestion and buffering for high-volume real-time data are better handled by a dedicated streaming delivery service that integrates with a Data Catalog for schema management.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed