"Claude can do this for me." It is a fair objection.

We hear it every week, usually from a sharp engineer who has watched Claude Code refactor a module in the time it takes to make coffee: my coding agent already reviews my code for security, so why would I pay for a pentesting product?

It deserves a real answer, not a sales one. Coding agents have become genuinely useful security reviewers. Anthropic ships a /security-review command in Claude Code, a multi-agent Claude Security plugin and a managed Claude Security product. OpenAI turned its Aardvark research agent into Codex Security. These tools find real bugs, and a team that ignores them is leaving easy wins on the table.

So the question is not whether coding agents help with security. They do. The question is narrower and more useful: what does each tool actually prove, and about which system? Once you see where each one's evidence ends, the overlap turns out to be much smaller than it looks.

The short version: a coding agent reads the blueprint. A pentest tries the doors.

Abstract Vulnix illustration: a code blueprint on one side, live attack paths reaching a running system on the other

What coding agents actually do for security today

It helps to be precise, because this landscape changed a lot in 2026. As of September:

  • Claude Code /security-review runs a single security pass over the changes on your current branch, from inside your terminal session.
  • The Claude Code security review GitHub Action runs the same analysis on pull request diffs. Its own README warns that it is not hardened against prompt injection and should only review trusted PRs.
  • The Claude Security plugin goes deeper. According to Anthropic's documentation, a team of agents maps your architecture, builds a threat model, hunts for vulnerabilities and has independent verifier agents review every finding before it reaches the report. It can then draft patches, which another agent reviews and tests before you apply them yourself.
  • Claude Security, the managed product, monitors connected repositories on the Enterprise plan and runs on Anthropic's Mythos-class models.
  • Codex Security, formerly Aardvark, builds a threat model of your repository, scans commits, attempts to trigger suspected bugs in an isolated sandbox and proposes patches.

That is a serious toolbox. It also has something in common: every one of these tools takes source code as its input. The unit of analysis is the repository, a branch or a diff.

Anthropic is candid about the consequences. Its documentation describes the plugin as one layer in a defense-in-depth stack, says it does not replace your existing source-code security tools, and notes that scans are nondeterministic: two scans of the same code can surface different findings.

None of that is a criticism. It is simply a description of what kind of tool this is.

Your repository is not your attack surface

Attackers do not attack your repository. They attack the thing you deployed, and the deployed system is made of far more than the code under review:

  • Configuration and secrets that live in environment variables, cloud consoles and CI settings, not in Git.
  • Infrastructure in front of the code: CDNs, load balancers, WAF rules, API gateways, storage bucket policies and CORS headers.
  • Identity: the actual auth provider setup, session lifetimes, OAuth redirect rules and the real permissions attached to real roles.
  • Version drift: the commit you reviewed is not always the build that is running. A fix can be merged while an old image keeps serving traffic.
  • Data and state: other tenants, existing records, feature flags and the sequence of steps a real user can take.

A coding agent can reason about some of this if you paste it into the context. It cannot observe it. When it says a route is vulnerable, it is making a prediction about a system it has never touched.

Left: reasoning about code. Right: testing the running system, then replaying the exploit after the fix.

This is the same gap the security industry has argued about for twenty years under the names SAST and DAST, static and dynamic testing. AI did not close it. It made both sides much more capable.

"Validated" means three different things

All three approaches now use the word validated, and the difference between them is the whole argument.

Approach What "validated" means Where the evidence comes from
Claude Security Independent verifier agents re-examine each finding and try to disprove it Reasoning over the source code
Codex Security The agent tries to trigger the bug in an isolated sandbox A build of your repository in OpenAI's sandbox
Vulnix An agent exploits the issue against the target you verified you own, and records the requests and responses Your running application, under an explicit scope

Each step down the table gets closer to the question a security decision actually depends on: can an attacker do this to us, today, against what we are running?

Here is what that looks like in practice. The finding below comes from a run in the public Vulnix demo workspace. Reading the code would tell you that the token API lacks an upper bound on expiry. The test shows more: the live API minted a token valid until the year 9999, the token authenticated successfully, and a control request confirmed the documented 90-day default still applies when no expiry is sent.

A Vulnix finding showing a write-up, a numbered proof of concept with a control step, severity metadata and a Validate Fix button.
A finding from the Vulnix demo workspace: each step of the proof of concept, including a control step, was executed against the deployed API.

That is the difference between "this looks unsafe" and "this is what happened when we tried it."

What the research says about AI reviewing code for bugs

The best public data comes from teams who measured it.

In September 2025, Semgrep's research team pointed Claude Code and OpenAI Codex at 11 large open-source Python web applications and reviewed more than 400 findings by hand. Claude Code's reports were true positives 14% of the time; Codex's, 18%. Both did best on logic-flavored bugs such as insecure direct object references, and struggled to trace injection flows across files. Most tellingly, running the same prompt three times on the same codebase produced 3, 6 and 11 distinct findings.

Models have improved a great deal since then, and verifier agents exist precisely to cut that noise. But the nondeterminism has not disappeared. Anthropic's own documentation still tells you to expect it.

There is a second pressure pushing the other way. Veracode's 2026 GenAI Code Security Report found that AI-generated code passed its security checks in only 56% of tasks, essentially unchanged from the year before, with cross-site scripting passing just 15% of the time. The same report estimates that AI now writes roughly half of all committed code.

Put those together and the picture is clear. More code is being written faster, by models that still introduce vulnerabilities, and reviewed by models whose findings vary run to run. That is an argument for more testing of the deployed result, not less.

Six questions a coding agent cannot answer from your repository

When a coding agent finishes a security review, these questions are still open:

  1. Is it reachable? Is the vulnerable route exposed to the internet in production, or blocked by a gateway, feature flag or network rule?
  2. Is it what you deployed? Does the running build match the commit that was reviewed?
  3. Do the controls hold across real identities? Does authorization actually stop user A from reading user B's data when both are real accounts in the live system?
  4. What does the chain yield? A low-severity leak plus a weak check can add up to account takeover. Chains only show up when you follow them.
  5. Did the fix work where it runs? A merged patch is not proof of remediation until the original attack fails against the deployed system.
  6. Can you show someone? Customers, auditors and your own leadership ask for evidence, not predictions.

A pentest exists to answer those questions. An AI pentest answers them more often.

"But Claude Code can run curl"

It can. Claude Code executes commands, and you could ask it to probe your staging URL. Some teams build impressive offensive setups this way.

The hard part of pentesting is not sending a request, though. It is everything around the request:

  • Authorization. Nothing stops a general-purpose agent from testing a domain you do not own. Vulnix requires proof of domain ownership before any run can start against a target.
  • Isolation. A coding agent attacks from your laptop, with your shell, your credentials and your network access. A Vulnix run executes in its own sandbox, created for that run and torn down afterwards, with private and internal network ranges blocked and test credentials scoped to the one run.
  • A record. Vulnix keeps the full agent trace, so you can see what each agent attempted, what it observed and why it concluded what it did.
  • Repeatability. When you fix a finding, Validate Fix replays the exact original exploit against that finding's asset and tells you whether it is confirmed fixed, still vulnerable or inconclusive.
The agent trace of a Vulnix pentest: 27 agents covering recon, authentication, injection, access control and business logic, each linked to the run that spawned it.
The agent trace from the same demo run: 27 specialist agents, each with its own messages and tool calls, all inspectable.

You could build all of that around a coding agent. At that point you would have built a pentesting product.

Where Vulnix fits, and where it does not

Vulnix is an agentic pentesting platform. It tests what attackers see:

  • Blackbox runs target a live web application, API or public IP range you have verified you own.
  • Whitebox runs clone a connected repository into the sandbox, and can attack a verified live target at the same time, so the agent can use the source to aim its attacks at the running system. See blackbox and whitebox testing.
  • PR reviews run a diff-scoped review of each pull request in an isolated sandbox and post it as a real GitHub review. See the GitHub integration.
  • Findings include the write-up, reproduction evidence, severity, CWE and remediation, and export into reports for the people who need them.

Just as important is what Vulnix is not. It is not a replacement for your coding agent's review, static analysis or dependency scanning. Those tools see every line of code; a pentest samples behavior within a scope. And our agents make mistakes too, which is exactly why every finding carries the evidence you need to check it yourself.

Use both, in the order they are good at

The strongest setup treats coding agents and pentesting as different stages of one loop:

Stage Best tool What you get
While writing code Your coding agent Fewer vulnerabilities written in the first place
Before merge /security-review, Claude Security or Codex Security, plus Vulnix PR review Suspected issues caught in the diff
After deploy A Vulnix pentest of the running app Proof of what is actually exploitable, with evidence
Fixing Your coding agent, fed the finding's Copy Fix Prompt A patch aimed at the demonstrated attack, not a guess
Closing Vulnix Validate Fix The original exploit replayed against the deployed fix

Notice the handoff in the middle. The two tools are not competitors in this workflow. Vulnix gives your coding agent something it rarely has: a confirmed, reproducible attack to fix against.

So, why pay for it?

Not for a smarter model. The models behind coding agents are excellent, and they keep getting better.

You pay for the parts a model does not provide: verified authorization to test a target, an isolated place to run the attack, execution against the system you actually deployed, evidence you can hand to someone else, and a replay that turns "we think it is fixed" into "the attack no longer works."

Use your coding agent to write better code. Use a pentest to find out what that code does once it meets the internet. The teams that ship safely in 2026 are the ones that do both.

Frequently asked questions

Can Claude Code do a penetration test?

Not in the sense a security team or auditor means. Claude Code, its /security-review command and the Claude Security plugin analyze source code and report suspected vulnerabilities. A penetration test exercises the deployed application under an authorization and scope, and shows what an attacker can actually do. Claude Code can run commands, but it does not bring target verification, an isolated execution environment, retained evidence or a repeatable retest.

Is Claude Security or Codex Security a replacement for a pentest?

No, and neither vendor positions it that way. Both are source-code security tools. Anthropic's documentation describes the Claude Security plugin as a layer alongside static analysis, dependency scanning and code review, and notes that scans are nondeterministic. Codex Security validates suspected bugs by triggering them in an isolated sandbox built from your repository, not against your production deployment.

What is the difference between AI code review and AI penetration testing?

AI code review reasons about what the code might do. AI penetration testing sends real, authorized traffic to the running system and records what it actually does. Code review sees every line but not the deployment; a pentest sees the deployment but samples behavior within a scope. Each catches problems the other cannot.

If AI writes most of our code, do we still need penetration testing?

Yes, arguably more than before. Veracode's 2026 GenAI Code Security Report found AI-generated code passed security checks in only 56% of tasks, while AI now authors roughly half of committed code. Faster code output means more change reaching production, and each change needs to be tested where it runs.

Can I use Claude Code or Codex together with Vulnix?

Yes, that is the intended workflow. Use a coding agent to write and review code, use Vulnix to prove what is exploitable in the running app, then paste the finding's fix prompt into your coding agent and use Validate Fix to replay the original exploit against the deployed fix.

Sources