Manual vs automatic reviews

ADO Pilot reviews automatically when a PR is created or updated. Developers can also start a review manually from the PR action menu. You decide which paths are available at the organization, project, or repository level.

The two trigger settings

reviewOnPush is scalar with override semantics — the most-specific scope wins. allowManualInvocation is honoured only at the organization level; setting it on a project or repository scope has no effect. See Org, project, and repo settings hierarchy.

Automatic reviews

When reviewOnPush is true, ADO Pilot starts a review whenever:

1. A pull request is created with a target branch matching your branch filter, or
2. A new commit is pushed to the source branch of an open PR.

A typical timeline:

13:00  Create PR from feature/login into main
13:01  ADO Pilot posts the AI review status check and inline comments
13:15  Push a new commit to feature/login
13:16  ADO Pilot re-reviews; the status check resets to pending, then resolves

To turn off automatic reviews, set reviewOnPush: false at the scope you want to affect.

Manual reviews

When allowManualInvocation is true, developers can start a review on any open PR using the PR action menu in Azure DevOps:

1. Open the pull request in Azure DevOps.
2. Click the More actions button (the three-dot menu) in the PR header.
3. Select Run AI Review.
4. The review starts immediately, or queues if your organization is at its concurrent-review limit.

A manual trigger bypasses some restrictions:

• It runs even if reviewOnPush is false.
• It does not bypass targetBranchFilters — the PR's target branch must still match a configured filter. (A future release will let manual reviews run on any branch; today they are gated by the same filter as automatic reviews.)
• It does not bypass enabled: false — a disabled repository cannot be reviewed manually.
• It does not bypass RC quota — if you are out of credits, manual reviews fail like automatic ones.

Common configurations

Default — automatic on every push, manual available on demand. Recommended for most teams.

reviewOnPush: true
allowManualInvocation: true

Manual-only — review when explicitly requested. Useful when a team wants to control spend tightly or only review just before merging.

reviewOnPush: false
allowManualInvocation: true

Auto-only — every push reviewed, no manual override. Rare; reduces flexibility for developers.

reviewOnPush: true
allowManualInvocation: false

Disabled. Equivalent to setting enabled: false on the repository.

enabled: false

How triggers interact with other settings

[^bypass]: A future release will let manual reviews bypass branch filters. Today the orchestrator applies the filter unconditionally.

Cost

Manual and automatic reviews consume RCs the same way. Triggering a review by hand does not make it free. For pricing details, see the billing documentation.

When to turn auto-review off

Most teams should leave reviewOnPush: true and use branch filters to narrow the set of PRs that get reviewed. Toggling auto-review off entirely makes sense in two cases:

• Tight cost control — A team that only wants the AI review just before merge, and is willing to require developers to click the action.
• Phased rollout — During the first week of an ADO Pilot trial, you may want manual-only so the team chooses when to engage with the tool.

For everyday operation, narrowing branch filters is a finer-grained tool than disabling auto-review.