When preparing CSV training data for a SageMaker built-in algorithm, converting the dataset to a numpy.array reduced training speed. What should you do to optimize training data for SageMaker built-in algorithms?

Converting the dataset to RecordIO protobuf format is the most effective way to optimize training data for SageMaker built-in algorithms. RecordIO is a highly efficient, optimized binary format designed for large-scale machine learning, offering faster data loading and processing compared to plain CSV or even numpy arrays. This format minimizes I/O bottlenecks, which directly translates to reduced training times. Using SageMaker batch transform to convert to a DataFrame is incorrect because DataFrames are in-memory structures and not an optimized storage format for large-scale training data on disk. AWS Glue compressing data into Apache Parquet is a good general data optimization, but RecordIO is specifically optimized for SageMaker's built-in algorithms and often outperforms Parquet for this specific use case. SageMaker hyperparameter optimization tunes model parameters, not the data format itself, so it wouldn't address the data loading bottleneck.