Migrating an on-prem Hadoop cluster to Dataproc would require 50 TB Persistent Disk per node and the CIO worries about block storage cost. How do you minimize storage cost for the migration?

Storing data in Google Cloud Storage (GCS) is the most cost-effective solution for large-scale data in Dataproc. GCS offers highly durable, scalable, and significantly cheaper object storage compared to Persistent Disk, especially for 50 TB per node. Dataproc clusters can directly access data in GCS, eliminating the need to store all data on expensive Persistent Disks attached to each node. Using preemptible VMs reduces compute costs, not storage costs. Tuning the cluster to have "just enough disk" is still using expensive Persistent Disk for all data, which is what the question aims to minimize. Migrating only cold data to GCS still leaves a substantial amount of data on Persistent Disks, which is less optimal than moving all data that doesn't require local HDFS storage to GCS.