A company is training a replacement model for one currently served by a SageMaker real-time endpoint. An ML engineer must measure the new model’s latency and accuracy under production traffic without impacting current users. Which evaluation approach satisfies this requirement?
Choose an answer
Tap an option to check your answer.
Correct answer: Perform shadow testing with a traffic sampling percentage of 100%..
Why this is the answer
Shadow testing (also known as dark launching) is the correct approach because it allows the new model to process 100% of production traffic in parallel with the existing model without affecting live users. The new model's predictions are recorded and can be analyzed for latency and accuracy offline, but its output is never returned to the end-user. This ensures no impact on current users while providing a realistic evaluation. Blue/green deployments with linear or canary traffic shifting involve gradually directing live user traffic to the new model, which would impact current users if the new model performs poorly. A rolling deployment also gradually replaces the existing fleet with the new model, similarly impacting users.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed