GitHub Lets AI Scan Pull Requests—So Why Can't It Block a Risky Merge?

GitHub’s AI Scan for pull requests is now generally available. It can inspect changed code, explain a suspected vulnerability, and sometimes propose an autofix. Yet the headline’s tension is real: GitHub explicitly says AI Scan findings are advisory, do not fail the code-scanning check, and cannot block a merge through repository rulesets.

That is not necessarily a defect. It is a boundary between a probabilistic reviewer and an enforcement mechanism. The problem begins when a team sees an AI security badge and assumes it has become a mandatory gate. The distinction matters because the two roles demand different properties: a reviewer can tolerate ambiguity and surface hunches, while a gate must produce the same verdict every time the same code arrives.

What AI Scan adds

AI Scan complements CodeQL by looking for vulnerabilities in pull-request changes, including languages and formats CodeQL may not currently cover. GitHub lists PHP, Shell and Bash, Terraform configuration, Dockerfiles, JSP for Java, and Blazor for C# among the supported examples.

When code scanning and the repository’s effective AI Scan setting are enabled, the system runs automatically on eligible pull requests. Findings appear beside CodeQL results in the conversation and changed-files views, marked as AI-generated. The result can include a risk explanation and a suggested fix.

This creates a useful second lens. Deterministic rules and semantic analysis remain strong at known vulnerability classes, while a language model can reason about surrounding intent and suspicious data flows that do not fit an existing query cleanly. The mechanism is worth understanding: the model reads the diff together with nearby context, forms a hypothesis about how untrusted input could reach a sensitive sink, and writes that reasoning out in prose. That prose is the product. It is also the reason the output cannot be treated as a verdict.

The coverage list is itself a signal about where the tool is aimed. Terraform configuration and Dockerfiles are infrastructure definitions rather than application logic, and mistakes there often hide in defaults, permissions, and image provenance rather than in a single function. Shell and Bash scripts rarely attract the same query depth as typed languages. PHP and JSP carry long histories of injection and templating flaws that reviewers may skim past. In each case the value is a second pass over change patterns that existing queries may not model.

Why it cannot block a merge

Security gates need predictable behavior. A check that blocks deployment must have a stable status, clear severity rules, an auditable exception process, and an acceptable false-positive rate. GitHub has chosen not to make AI Scan findings satisfy that role.

The official announcement says the alerts do not fail the code-scanning check. Repository administrators therefore cannot add a ruleset requirement that automatically prevents merging solely because AI Scan reported a finding.

The conservative design limits damage from hallucinated or context-poor findings. An AI reviewer may misunderstand a sanitizer, ignore configuration outside the diff, or flag code that is unreachable. Automatically freezing every pull request on such evidence could turn security tooling into a denial-of-service mechanism against development. Consider the failure mode in detail: a model that cannot see the framework’s global middleware may believe a parameter is unsanitized, and a blocking check would then demand action on a finding that is simply wrong. Multiplied across a busy repository, that friction teaches developers to bypass the tool.

But the same design creates a human-process risk. A severe-looking finding can remain visible while someone with merge authority proceeds anyway. Teams must decide who triages the alert and which independent control will enforce the final decision. Visibility without ownership is the quiet failure mode here: the warning is rendered, the reviewer is busy, and the merge button stays green.

Important coverage gaps

AI Scan does not run on pull requests from forks or those created by Dependabot. That matters for open-source projects, which commonly receive outside contributions through forks, and for repositories that rely on automated dependency updates.

The fork exclusion deserves emphasis because it inverts the usual expectation. The pull requests a maintainer knows least about, arriving from strangers, are precisely the ones the AI reviewer will not see. Dependency-update PRs are equally consequential: a version bump can change transitive behavior, yet the automated change bypasses the scan. Maintainers should treat these paths as requiring the deterministic scanners and human review they already trust.

The feature is available on public repositories owned by people or organizations. Private and internal organization repositories require GitHub Code Security or GitHub Advanced Security. A Copilot license is not required, but the scans consume Code Security AI Credits, making budgets and usage monitoring part of deployment planning. That consumption model turns security review into a metered resource, so teams should decide in advance which repositories warrant the spend rather than enabling it everywhere and discovering the cost later.

AI Scan is also not described as comprehensive. It should not replace dependency scanning, secret scanning, CodeQL, conventional tests, manual review, or runtime controls. A clean AI report means only that this tool did not surface an advisory finding on the eligible change. Absence of evidence, in other words, is not evidence of absence.

How teams should use it

Start with a defined triage policy. Decide who owns AI alerts, how quickly high-risk findings must be reviewed, and what evidence is needed to dismiss one. The reviewer should inspect the full data flow and relevant configuration, not merely accept or reject the generated explanation. A dismissal recorded with a reason becomes institutional memory; a dismissal made silently becomes a recurring argument.

If a finding is validated, convert it into an enforceable control: a CodeQL query, static-analysis rule, regression test, policy check, or required human approval. This turns a probabilistic discovery into a repeatable guardrail for future pull requests. The pattern generalizes: every confirmed AI finding is a lesson about a class of bug, and the durable response is to encode that lesson where deterministic tooling can enforce it.

Teams should also track usefulness. Measure confirmed findings, false positives, duplicate alerts, time to triage, and vulnerabilities found by other tools after AI Scan passed. Without those metrics, AI review can create activity without proving risk reduction. The comparison against other tools is the sharpest test, because it reveals what the AI lens caught that the deterministic stack missed, and what slipped past both.

What developers should not assume

Do not assume an AI-suggested autofix is secure merely because it compiles. The patch may move the vulnerability, break expected behavior, or create a new edge case. Treat it as a proposed change that requires the same tests and review as code written by a person. An autofix that adds an escaping call in the wrong layer, for instance, can satisfy the compiler while leaving the original data path exposed.

Do not assume “generally available” means eligible for every repository or PR source. Fork and Dependabot exclusions should be visible in the team’s security documentation so maintainers know which changes need alternative checks.

Finally, do not confuse advisory findings with branch protection. If a repository needs blocking security checks, keep deterministic scanners and required reviews in the ruleset.

Verdict

GitHub AI Scan is best understood as a new source of evidence, not a security gate. Its value lies in extending review into languages and change patterns that existing queries may miss, then giving humans a faster starting point for investigation.

The inability to block merges is deliberate and defensible, but it transfers responsibility back to the organization. A mature team will use AI Scan to discover risks and then encode confirmed lessons into enforceable tests and policies. A careless team may display more warnings while continuing to merge exactly as before.