Setting AI review as a required PR check

ADO Pilot posts a status check on every PR it reviews. You can promote that check into a required branch policy in Azure DevOps so PRs cannot merge until the AI review completes (and, optionally, passes). This page shows you how to wire the policy.

The status check identifier

ADO Pilot's status check uses these exact identifiers. You will look for them by name when you add the branch policy.

The check appears in the Checks section of every PR alongside any CI/CD pipeline checks.

Status states

The check transitions as the review progresses.

Both passed and advisory outcomes report succeeded — the check only blocks merge when the AI flags a true blocker (or the system errors).

Add the policy in Azure DevOps

You configure required status checks per branch in Azure DevOps. The walkthrough below targets main; repeat for any other branch you want to protect.

Step 1 — open branch policies

1. In Azure DevOps, open your project.
2. Go to Repos > Branches.
3. Locate the branch you want to protect (for example, main).
4. Hover over the branch row and click the More options button (three dots).
5. Click Branch policies.

Step 2 — add the status check

1. Scroll to the Status checks section of the branch policy page.
2. Click the + button (Add status check).
3. In the Status to check dropdown, search for adopilot/ai-pr-review. If you do not see it, the search also matches ai-pr-review.
4. Select the status. The full context adopilot/ai-pr-review should populate the field.

Step 3 — choose the policy requirement

1. Policy requirement — Choose Required to block merge until the check passes. Choose Optional to display the check without enforcing it.
2. Authorized users / groups — Leave default unless you want to restrict who can post or override this check. ADO Pilot's service principal is already authorized to post the status; you do not need to add it manually.
3. Policy applicability — Choose Apply only to PRs that have changes affecting these paths if you only want the gate on certain paths; otherwise leave default.

Step 4 — choose the reset behavior

Under the policy, set Reset status whenever there are new changes (called Reset on new pushes in some Azure DevOps versions). When set, every new commit invalidates the previous status, and the merge button stays disabled until ADO Pilot posts a fresh succeeded result. Recommended.

Step 5 — save

Click Save to apply the policy. The branch is now gated on AI review.

Step 6 — verify

1. Open or create a PR targeting the protected branch.
2. Confirm the AI Review check appears under Checks in the PR.
3. Confirm the merge button is disabled while the check is pending or failed.
4. After the review completes successfully, confirm merge becomes available (assuming all other required policies pass).

Soft gate vs hard gate

A common rollout pattern:

• Week 1–2 — soft gate. Add the status check as Optional. The team sees the AI's findings on every PR but is not blocked from merging.
• Week 3+ — hard gate. Switch the policy to Required. PRs now cannot merge until the AI review reports succeeded.

The soft-gate phase gives the team a chance to tune branch filters and file exclusions before merge gates depend on the AI's output.

Override and bypass

If the policy is set to Required, the merge is blocked when the check is failed or error. Two ways to merge anyway:

• Bypass — A user with the Bypass policies when completing pull requests permission can merge without the check passing. Bypasses are recorded in the PR audit log.
• Re-trigger — Address the AI's findings, push a fix, and let ADO Pilot re-review. With Reset on new pushes enabled, the new commits invalidate the failing status.

Removing the policy

To stop gating on AI review without disabling the AI review itself:

1. Open the branch policy page (Step 1 above).
2. Find the AI review status check entry.
3. Click the Remove button.

Removing the branch policy does not stop ADO Pilot from posting the status check — it only stops the check from blocking merge. To stop reviews entirely, set the repository to enabled: false. See Choosing which repositories get reviewed.

Troubleshooting

The status check is not in the dropdown. Azure DevOps only lists status contexts that have been posted on the repository before. Open one PR so ADO Pilot posts an initial status, then return to the policy page.

The check is pending and never finishes. Verify the repository has reviews enabled and the PR's target branch matches your branch filters. See Targeting branches. If both are correct, check the customer dashboard for a service-side error.

The check passed, but I still cannot merge. Other required policies may still be blocking — check for a required code-reviewer count, a required build, or a work-item link policy.

The check resets every time I push a fix. That is the Reset on new pushes behavior working correctly. Each new commit triggers a new review; the merge unlocks again once the new review reports succeeded.