A team adds an evaluation workflow for agent-generated pull requests. The workflow should preserve enough evidence for error analysis even when the agent passes some checks but fails others. Evaluate the following statements. 1. checks.json provides quantitative workflow status evidence. 2. review-signals.json alone proves the task is successful. 3. if: always() helps retain evidence after failed steps.
name: agent-evaluation
on:
pull_request:
branches:
- main
permissions:
contents: read
pull-requests: read
checks: read
jobs:
evaluate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Collect test output
run: |
mkdir -p evaluation
npm ci
npm test 2>&1 | tee evaluation/test-output.log
- name: Collect check state
env:
GH_TOKEN: ${{ github.token }}
run: |
gh pr checks ${{ github.event.pull_request.number }} \
--json name,state,conclusion \
> evaluation/checks.json
- name: Collect review signals
env:
GH_TOKEN: ${{ github.token }}
run: |
gh pr view ${{ github.event.pull_request.number }} \
--json reviews,comments \
> evaluation/review-signals.json
- name: Upload evaluation evidence
if: always()
uses: actions/upload-artifact@v4
with:
name: agent-evaluation
path: evaluation/Choose an answer
Tap an option to check your answer.
Correct answer: 1=Yes 2=No 3=Yes.
Why this is the answer
Statement 1 is correct because checks.json captures the name, state, and conclusion of all checks on the pull request. This provides quantitative data on which checks passed or failed, offering concrete evidence of the workflow's status. Statement 2 is incorrect because review-signals.json contains human reviews and comments, which are qualitative feedback. While valuable, they don't definitively prove task success; a task might pass all automated checks but still have negative human feedback, or vice-versa. Statement 3 is correct because if: always() ensures the "Upload evaluation evidence" step runs regardless of whether previous steps succeeded or failed. This is crucial for retaining diagnostic information (like test-output.log, checks.json, and review-signals.json) even when the agent's generated code causes test failures, enabling comprehensive error analysis.
Pass your exam — without the endless answer hunt
Get every verified question and explanation for this exam in one place, and save hours of prep. 1,000+ certifications · 20+ languages · free to start.
Pass your exam faster → No card needed