What is not reviewed

The boundary cases ADO Pilot skips — large PRs, binaries, excluded files, target-branch filters, closed PRs, and suspended tenants.

Last updated

ADO Pilot skips some pull requests and some files inside otherwise-reviewable PRs. Knowing which is which helps you understand why a PR did not get a review, why the comment looks empty, and when a PASS does not actually mean "we looked at everything."

What gets skipped, at a glance

BoundaryBehaviorCharged review credits?
Diff over 5,000 changed linesReviews the first files up to the cap, skips the rest, posts a partial result.Yes, for the lines reviewed. Not for the skipped tail.
Binary files (images, archives, compiled output)Filtered before review. Never sent to the model.No.
Files matching exclusion patternsFiltered before review. Same effect as binaries.No.
PR with every changed file excludedPosts a PASS with zero findings and "no reviewable files."No.
Target branch does not match targetBranchFiltersWebhook fires but the orchestrator skips the review.No.
PR is closed, merged, or abandonedNew reviews are rejected.No.
Tenant suspended or cancelledWebhooks accepted and recorded; new reviews rejected. Reviews already past quota reservation finish normally.No, except for in-flight reviews that already reserved quota.
enabled: false at org, project, or repoWebhook fires but the orchestrator skips the review.No.

The rest of this page expands each row with the practical detail.

Large PRs — the 5,000-line cap

A single review is capped at 5,000 changed lines (additions plus deletions, after exclusions are applied). When a PR exceeds the cap:

  • ADO Pilot reviews files in deterministic order until the running line total reaches the cap.
  • Remaining files are skipped. They are not reviewed at all — not "skimmed lightly."
  • The tracking comment notes the partial review explicitly: Reviewed 30 of 50 files; 20 files skipped due to size.
  • You are charged review credits only for the lines actually reviewed, not for the skipped tail.

Workarounds: split the PR into smaller change sets, or add the bulk-changed paths (lockfiles, generated code, vendored dependencies) to your exclusions so they do not eat the budget.

Binaries and irreviewable formats

Binary diffs do not contain text the reviewer can analyze, so they are filtered before review:

  • Images: .png, .jpg, .jpeg, .gif, .webp, .svg, .ico
  • Archives: .zip, .tar, .gz, .7z, .rar
  • Compiled artifacts: .exe, .dll, .so, .o, .a, .lib, .wasm
  • Media: .mp3, .mp4, .wav, .mov, .avi
  • Documents: .pdf, .docx, .xlsx, .pptx

These are silently skipped. They do not consume review credits and they do not appear in the finding count.

Excluded files

You can add your own glob patterns at three levels — org, project, or repo — under Settings → Repositories. Patterns merge across levels (the more specific scope adds to the less specific scope; nothing is overridden away).

The default exclusions, applied to every new repo:

  • **/package-lock.json, **/yarn.lock, **/pnpm-lock.yaml and other lockfiles
  • **/*.generated.* and other generated source
  • **/*.min.js, **/*.min.css
  • obj/**, dist/**, build/**, out/** build output trees

Glob syntax is the standard form: ** matches any path segment, * matches anything inside a single segment, ? matches one character, and [abc] matches a character class.

A PR where every changed file matches an exclusion pattern is a special case. ADO Pilot still posts a tracking comment, but the comment shows PASS with zero findings and no review credits are consumed. This is intentional — there is nothing to review, and treating an all-excluded PR as a failure would be misleading.

Target branch filtering

Each repo can specify which target branches should trigger reviews using glob patterns — for example, ["main", "release/*"]. PRs whose target branch does not match are skipped:

  • The webhook still arrives at our backend.
  • The orchestrator records that the event was received and then drops it.
  • No review runs, no comment posts, no review credits are charged.

Use this to keep draft branches and exploratory feature-branch chains out of the review flow.

PR lifecycle states

Reviews only run on open PRs. If a PR is merged, abandoned (closed without merging in Azure DevOps terminology), or otherwise closed:

  • The git.pullrequest.updated webhook is still received.
  • The orchestrator detects the closed state and rejects the review.
  • A manual "Run AI Review" attempt against a closed PR is also rejected.

To get a review on a closed PR, reopen it first. See Re-running a review for the manual trigger options.

Tenant state — suspended or cancelled

If your ADO Pilot subscription is in a suspended or cancelled state — for example, a billing failure or an admin-initiated cancellation — new reviews stop:

  • Webhooks are still accepted and stored so we can replay them after recovery.
  • New reviews are rejected while the tenant is suspended.
  • Reviews that had already reserved a review credit before suspension finish normally — the orchestrator does not abandon work the customer has already paid for.
  • No review credits are consumed for the new, rejected webhooks.

The admin portal and the Overage behavior page show the suspension state and the steps to restore it.

Disabled at any scope

The enabled setting can be flipped off at the org, project, or repo level. The most-specific setting wins, and the resolved value gates whether the orchestrator runs the review at all. A repo with enabled: false produces no reviews even if the org and project are enabled.

What never gets flagged inside a reviewed file

A few categories of finding are intentionally suppressed even when the file is in scope:

  • Pre-existing issues in unchanged lines. ADO Pilot reviews the diff, not the whole file.
  • Mechanical formatting (indentation, brace placement, line length).
  • Issues a linter already catches, unless they signal a real bug.
  • Subjective preferences when both forms are correct (const vs let, early-return vs nested).
  • Performance concerns in test fixtures and cold administrative paths.
  • TODOs and FIXMEs the author already left as comments.

See Severities and categories for the positive scope — what does get flagged.