Frequently asked questions

Short answers to the ten questions we hear most often. Each one links to a deeper page if you want the full picture.

What is a Review Credit (RC), and how many will my PR cost?

One RC ≈ 500 changed lines (additions + deletions, after binary/exclusion filtering, with long lines normalized at 200 chars). Empty diffs are free. A typical "medium PR" is ~1–2 RCs; that's where the "~75 medium PRs/month" copy on the billing page comes from. See Plan tiers for the included RC count for each plan.

Learn more: Review credits explained

How do reviews actually appear on my pull request?

Two surfaces: (a) inline comments threaded to specific file:line ranges, each with a severity emoji + category + optional one-click suggestion code fence; (b) one PR-level summary comment that updates in place (queued → in progress → complete) and ends with a PASS, ADVISORY, or FAIL verdict. There's also a status check (ai-pr-review) you can wire up as a required branch policy.

Learn more: Anatomy of a review comment · Setting AI review as a required PR check

What happens when I run out of credits?

Depends on your plan's overage policy. Hard-block tiers (Starter/Team default) reject new reviews silently — no PR status check or comments appear, the dashboard's Reviews page shows the review with status Failed and reason Quota exceeded, and your org admin gets a Review quota exhausted email. No credit is charged for the rejected review. Auto-overage tiers (Business/Enterprise default) keep reviewing and bill the overage rate per extra RC, capped at the "Overage Cap" you set on the billing page. Quota resets on your Stripe billing anniversary, not the 1st of the month.

Learn more: Review quota exceeded · Hard-block vs auto-overage

Do I have to install the Marketplace extension for reviews to run?

No. Reviews fire automatically once the wizard provisions the service-hook subscriptions in your ADO org. Installing the extension adds two niceties: a "Run AI Review" button in the PR action menu, and a Project Admin settings hub. Most config still lives at adopilot.dev.

Learn more: Installing the Marketplace extension (optional) · Using the PR action menu

What permissions / scopes does the PAT need, and why?

ADO Pilot needs the following Azure DevOps personal access token (PAT) scopes. Pick exactly these — do not grant Full access.

ADO Pilot's onboarding wizard probes the Code (Read) scope at validation time. The other three scopes (Code Status, Pull Request Threads, Service Hooks) are exercised when the wizard provisions service hook subscriptions and when ADO Pilot posts the first review. A PAT missing any of them will fail later in onboarding or at first review with a missing-scope error — re-create the token with all four scopes selected if you see one.

If you create a PAT with anything missing, the wizard returns a pat_missing_scope error pinpointing which scope is absent.

Learn more: Creating a Personal Access Token · PAT and onboarding wizard error codes

What does ADO Pilot send to Anthropic, and is my code retained?

Only the diff content and filenames go to Anthropic; we don't ingest your full repo. We use Anthropic's Batch API as the primary path (50% cheaper) with the Messages API as fallback. Cached prompt context has short TTLs (instructions 1h, diff 5m). See the Privacy policy and DPA for the formal data-flow description and retention windows.

Learn more: Data handling — what we send to Anthropic · Privacy policy and DPA

Why do I sometimes see findings in the "in progress" comment that disappear from the final review?

That's the two-pass design working as intended. Pass 1 over-reports to maximize recall; Pass 2 re-evaluates each finding, rescinds false positives (typically 30–50%), and writes the final user-facing comments. Only the survivors are posted as inline threads.

Learn more: Why the review is two passes (and why some findings disappear)

What files does ADO Pilot skip, and how do I add my own exclusions?

Always skipped: binaries, anything matching the org/project/repo fileExclusionPatterns (defaults catch lock files, generated code, minified bundles), and any individual file in a diff bigger than the size threshold. Excluded lines never count against your RCs. Add patterns at Settings → Repositories (org-wide) or per-repo. Filters are additive across the inheritance chain.

Learn more: Excluding files from review · What ADO Pilot won't review (and why)

A teammate updated the PR while a review was running — what happens?

The orchestrator detects the new iteration, abandons the in-flight review, and starts a fresh one against the latest commit. You'll see the summary comment update in place rather than two competing comments. The abandoned review doesn't count against your quota.

Learn more: Mid-review pushes — which commit gets reviewed?

How do I stop reviews running on a specific repo or branch?

Three knobs, narrowest first: at the repo level, toggle off the repo in Settings → Repositories; at the project level, set targetBranchFilters to the branches you actually want reviewed (e.g., main, release/*); at the org level, you can set the master enabled switch off, but that disables ADO Pilot for everyone. Status-check requirements set in ADO branch policies are independent — turning off reviews doesn't auto-remove the policy.

Learn more: Choosing which repositories get reviewed · Targeting branches