A neural network trained with stochastic gradient descent shows high, oscillating training and validation loss that decreases for a few epochs and then increases repeatedly. What action should the ML engineer take to stabilize and improve training?

High, oscillating loss that decreases and then increases repeatedly is a classic sign of an unstable training process, often caused by a learning rate that is too high. A large learning rate causes the model to overshoot the optimal minimum in the loss landscape, leading to oscillations and preventing convergence. Decreasing the learning rate allows for smaller, more controlled steps, helping the model to converge smoothly to a minimum and stabilize training. Applying early stopping would halt training prematurely but wouldn't address the underlying instability. Increasing the test set size is for evaluation, not training stability. Increasing the learning rate would exacerbate the problem, making the oscillations worse.