A data scientist trains a large PyTorch model on SageMaker; each training job takes about 10 hours. They suspect training isn’t converging and that GPU utilization might be poor. Which approach requires the least development effort to detect and address these training issues?
Choose an answer
Tap an option to check your answer.
Correct answer: Use SageMaker Debugger’s built-in rules such as vanishing_gradient and LowGPUUtilization to detect problems and trigger the StopTrainingJob action..
Why this is the answer
SageMaker Debugger is designed for exactly this scenario. Its built-in rules, like vanishinggradient and LowGPUUtilization, directly address the suspected issues of non-convergence and poor GPU utilization. It integrates seamlessly with SageMaker training jobs and can automatically trigger actions like StopTrainingJob, requiring minimal development effort. Using CPU utilization metrics in CloudWatch is less direct for GPU-bound training and might not accurately reflect GPU performance or convergence issues. Publishing high-resolution custom metrics to CloudWatch and using a Lambda function would require significant development effort to implement the metrics, analysis logic, and job stopping mechanism. The rules confusion and featureimportanceoverweight are not standard built-in Debugger rules for detecting vanishing gradients or low GPU utilization; they are not relevant to the problem description.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed