You receive average prices for 100 common goods every 30 minutes and need them up to date in BigQuery for cheap joins. Which approach is best?

The best approach is to store and update the data in a regional Cloud Storage bucket and create a federated BigQuery external table. This method ensures the data is always up-to-date in BigQuery without requiring repeated data loading. Federated queries allow BigQuery to query data directly from Cloud Storage, providing "cheap joins" as requested. Loading data every 30 minutes into a new partitioned table would create many small tables and incur unnecessary storage and loading costs. Storing data in Cloud Datastore or a Cloud Storage file and using Cloud Dataflow for programmatic merging would introduce additional complexity, latency, and cost compared to the direct federated query approach for this use case.