A team evaluates Copilot cloud agent over 30 issue-assignment runs. The agent usually creates a draft pull request, but review feedback shows repeated violations in generated API changes. Which change should you make first? Select one answer.
Evaluation: agent-api-change-suite
Runs: 30
Pass rate: 63%
Recurring findings:
- 18 PRs modified generated OpenAPI client files directly
- 14 PRs skipped pnpm test:contracts
- 11 PRs changed src/api/routes/** without updating tests/contracts/**
- 9 PRs passed unit tests but failed contract tests after review
Current repository guidance:
.github/copilot-instructions.md:
- Use pnpm.
- Keep code readable.
- Add tests when useful.
Current validation workflow:
on:
pull_request:
branches: [main]
jobs:
unit:
steps:
- run: pnpm test:unitChoose an answer
Tap an option to check your answer.
Correct answer: Add explicit contract instructions and checks.
Why this is the answer
The repeated violations in API changes, specifically modifying generated OpenAPI client files directly, skipping pnpm test:contracts, and changing src/api/routes/ without updating tests/contracts/, strongly indicate a lack of understanding or adherence to contract testing and API design principles by the Copilot agent. Adding explicit contract instructions to .github/copilot-instructions.md and integrating contract test checks into the validation workflow will directly address these recurring issues by guiding the agent and enforcing proper behavior. Increasing the model context size might not solve the problem if the agent isn't explicitly instructed on contract best practices. Converting draft pull requests to ready is premature and risky given the quality issues. Adding a schedule for nightly validation is a reactive measure and won't prevent the agent from making the same mistakes in the first place.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed