NEWS 9 min read

Can Anthropic Really Stop 99% of Attacks That Steal Claude's 'Thoughts'?

Anthropic says new defenses cut reasoning-data extraction by 90–99%. The number is promising, but it is an internal measurement—not proof that model extraction is solved.

By EgoistAI ·
Can Anthropic Really Stop 99% of Attacks That Steal Claude's 'Thoughts'?

Anthropic says its new safeguards reduce the amount of useful Claude reasoning data harvested by model-distillation attacks by 90% to 99%. Could an ordinary developer lose access simply for making many API calls? Not merely for volume, according to the company’s description—but automated detection, blocked accounts, and false positives now form a more visible part of the platform boundary.

The strongest numbers are Anthropic’s own measurements. The company reports a false-positive rate below 0.1%, but independent reproduction is not yet available. Developers should treat the defense as meaningful security progress and as a new operational risk to monitor.

What attackers are trying to steal

Model distillation is a legitimate technique when a developer has permission to use the teacher model’s outputs for training. It becomes model extraction or theft when an attacker sends industrial-scale queries to reconstruct capabilities without authorization.

Reasoning traces are especially valuable because they contain intermediate steps, corrections, decompositions, and decision patterns. A final answer provides one label. A detailed trace can teach a smaller model how to approach an entire class of problems.

Anthropic cites an internal experiment in which a model trained with fuller reasoning data reached 80.4% on SWE-bench Verified, compared with 71.8% when trained on summarized reasoning. The exact result depends on the experiment, but it illustrates why providers increasingly treat reasoning records as proprietary training data.

The asymmetry is what makes the economics work for attackers. A single query returns one answer; a single query that also returns a trace returns a worked example of how to get there. If a competitor can harvest millions of those worked examples cheaply, it can approximate a model’s behavior on new problems without paying to train the original from scratch. That is why the trace, not the answer, is the asset worth defending—and why providers have moved from treating reasoning as a debugging courtesy to treating it as inventory.

How Anthropic says the defense works

The system combines several layers rather than relying on a single rate limit:

  1. Preserved thinking: users receive useful reasoning summaries without the complete internal trace.
  2. Access separation: reasoning access is restricted across models and organizations.
  3. Two-stage classification: traffic patterns and sampled content are evaluated for extraction behavior.
  4. Linked-account enforcement: repeated campaigns cannot reset easily by creating another account.
  5. Operational blocking: suspected extraction requests and accounts can be denied.

Anthropic says these controls reduced harvested reasoning tokens by one to two orders of magnitude and led to the blocking of tens of thousands of accounts.

Each layer addresses a different weakness in the previous one. Summaries alone would be defeated if attackers could request the full trace through a different endpoint, so access separation closes that door. Classification alone would be defeated by volume, so the two-stage design samples content as well as counting requests. Enforcement alone would be defeated by account churn, so linking accounts removes the cheapest escape hatch. Blocking is the last resort, not the only tool—which is the part legitimate developers should care about most, because it means most traffic never reaches the enforcement stage at all.

Does heavy API use put legitimate users at risk?

High volume alone is not the same as extraction. Normal products can generate millions of repetitive classification, coding, or evaluation requests. The risk rises when traffic resembles systematic capability harvesting: broad prompt suites, repeated reasoning elicitation, coordinated accounts, proxy rotation, and training-oriented collection patterns.

Still, any classifier can make mistakes. Anthropic acknowledges integration costs, false positives, and the difficulty of evaluating adaptive attackers. A reported rate below 0.1% may be small statistically while still affecting real customers at scale.

Teams that depend on Claude should retain request IDs, document their permitted use case, avoid undocumented account sharing, and maintain an escalation path for unexpected blocks. They should also understand whether denied requests are billed and how retries affect cost; policy and billing details can change independently of the research result.

The practical lesson is that behavioral security systems reward legibility. A team whose traffic looks like a coherent product—steady prompt distributions, identifiable use cases, stable account structure—is easier to classify as legitimate than a team whose traffic looks like a scraper with a business license. Documentation is not bureaucracy here; it is evidence that a classifier or a human reviewer can use when a block is contested.

Security versus developer transparency

Reasoning restrictions create a genuine tradeoff. Hiding detailed traces makes theft harder, but it can also reduce debugging, reproducibility, and auditability. A summary may explain the broad approach while omitting the exact step that caused an error.

Providers can reduce this tension with structured tool logs, citations, deterministic test modes, appeal processes, and organization-level audit records. Customers do not necessarily need raw chain-of-thought, but they do need enough evidence to understand consequential failures.

This is the counterweight to the headline number. A defense that protects data by making failures harder to diagnose trades one kind of risk for another, and the trade is only acceptable if the replacement evidence is good enough. Structured logs and citations are not as rich as a full trace, but they are auditable, reproducible, and far harder to harvest at scale—which is precisely why they may become the standard compromise.

Is 99% the end of model extraction?

No. The number describes the reduction observed under Anthropic’s evaluated attacks and defenses. Adaptive adversaries can change prompts, distribute traffic, target final answers instead of reasoning, or move to compromised customer accounts.

Defense also changes the economics rather than making theft mathematically impossible. If extracting the same signal requires 10 to 100 times more accounts, queries, and money, many campaigns become less attractive even when some data still escapes.

That framing matters for how readers should interpret the result. A 90–99% reduction is not a wall; it is a tax. Taxes change behavior at the margin, and the campaigns most likely to disappear are the marginal ones—the side projects, the budget-constrained startups, the experiments that were only worth running while extraction was nearly free. Well-funded adversaries will keep paying, which is why the number should be read as a shift in incentives rather than a final victory.

Verdict

Anthropic’s result shows that reasoning data has become a protected asset, not merely an explanatory by-product. The new controls appear capable of sharply raising the cost of large-scale extraction.

Legitimate developers are not being told that heavy usage automatically means account termination. But they are operating inside a stricter behavioral security system. That makes clear policies, observable enforcement, billing transparency, and a reliable appeal process as important as the headline 90–99% reduction.

Share this article

> Want more like this?

Get the best AI insights delivered weekly.

By subscribing, you agree to our Privacy Policy. You can unsubscribe at any time.

> Related Articles