On-prem servers upload to Cloud Storage over a 10 Gbps Direct Peering link with 100 ms RTT and uploads do not saturate the link. What should you do on the on-prem servers?

The 100 ms RTT (Round Trip Time) indicates high latency, which significantly impacts TCP throughput, especially over high-bandwidth links like 10 Gbps. TCP's windowing mechanism limits the amount of data in flight, and high latency means it takes longer for acknowledgments to return, causing the window to fill slowly and underutilize the available bandwidth. Tuning TCP parameters, such as increasing the TCP window size (e.g., using TCP window scaling), allows more data to be sent before an acknowledgment is required, effectively mitigating the latency bottleneck and saturating the link. Compressing files might reduce data size but doesn't address the underlying TCP performance issue caused by high latency. Removing the -m flag from gsutil disables parallel uploads, which would decrease performance. The perfdiag parameter is for diagnostics, not for enabling faster performance.