Copilot coding agent opened a draft pull request for an issue that required two outcomes: Preserve the public REST API contract. Add a regression test for the reported bug. The PR summary says both were completed, but the evaluation workflow failed. Which option best explains the evaluation result?
Workflow: evaluate-agent-task
Event: pull_request
Actor: github-copilot[bot]
Run npm run test:contract
✓ API contract tests passed
Run npm run test:regression
FAIL tests/regression/payment-timeout.test.ts
Expected: timeout error is returned with retry=false
Received: timeout error is returned with retry=true
Run npm run report:evaluation
Evaluation report:
contract=passed
regression=failed
required_outcome=not_met
Conclusion: failureChoose an answer
Tap an option to check your answer.
Correct answer: The expected regression outcome was not met.
Why this is the answer
The evaluation workflow clearly states regression=failed and requiredoutcome=notmet, indicating that the regression test did not pass as expected. The test output further specifies the failure: "Expected: timeout error is returned with retry=false" but "Received: timeout error is returned with retry=true." This shows the regression bug was not fixed according to the requirements. The PR did not fail because Copilot authored it; the evaluation is based on code quality, not author. The API contract check ✓ API contract tests passed indicates it did not block testing. The workflow ran successfully, as evidenced by the detailed output, so it was not skipped due to path filters.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed