AI Pentest Agents Part 1: From Assistant to Autonomous Swarm

I read about PentestCode on 17 July 2026 — a hard fork of OpenCode rebuilt specifically for offensive security, with thirteen coordinating subagents and a persistent engagement-state graph. I went looking for what else had shipped recently and found it wasn’t an isolated release. In the two weeks either side of that piece, the same outlet covered KittySploit (1,150+ modules), Xalgorix (a 22-phase autonomous methodology), and T3MP3ST (turning general-purpose coding agents into bug hunters with 35 tools). That’s four AI pentesting tools launched inside a month, on top of an already-established field.

This series is about that field: what these tools actually do under the hood, how they differ from each other, and — since this site has already run its own experiment in this exact space — how they differ from what I built in AI Part 7, where I wired a local LLM into llm-tools-nmap and MetasploitMCP and watched a 7B model fabricate an entire successful exploit transcript out of nothing.

Three ways to wire an LLM into offensive security

Reading through the current field, there are really three distinct architectures in play, not one continuum.

Assistant-style tools put a human in the driving seat and an LLM in the passenger seat. The original PentestGPT, published at USENIX Security 2024, worked this way: a tester describes what they’re looking at, the model reasons about next steps, the tester decides what to actually run. HackerAI and similar copilot-style tools sit in the same category — useful for triage and report-writing, but nothing acts without a person approving the action first.

Autonomous multi-agent swarms flip that relationship. You give the system a target and a goal — “10.10.10.5, domain admin” — and a coordinator agent decomposes that into tasks, spawns specialist subagents (recon, enumeration, exploitation, post-exploitation), and the agents execute, observe results, and decide the next move themselves. PentestCode works this way. So does PentAGI, an open-source system with over 10,000 GitHub stars that runs its agents — Coder, Pentester, Installer, Memorist — inside sandboxed Docker containers pre-loaded with more than twenty tools. So, in a more polished commercial form, does Horizon3.ai’s NodeZero, and so does XBOW, which became the first autonomous system to top HackerOne’s global leaderboard.

Raw tool-calling is the third, narrower shape, and it’s what I actually built in Part 7. There’s no coordinator, no subagents, no persistent engagement-state graph, no shared memory between tool calls beyond the model’s own context window. A single general-purpose LLM gets handed a flat list of functions — nmap_scan, run_exploit, send_session_command — and decides which one to call next, with nothing tracking what it’s already tried or what it’s already found except its own conversation history. It’s the same underlying idea (natural language in, real function call out, real output back) with none of the orchestration scaffolding the swarm-style tools build on top.

That third category matters for this series because it’s a genuine baseline, not a strawman. It’s what you get if you reach for an LLM and a couple of tool wrappers without adopting one of the frameworks above — and Part 7 already surfaced the sharpest problem with it: a model is not a trustworthy narrator of its own actions. A 7B model in that experiment didn’t just get an answer wrong, it invented a complete, plausible-looking successful exploit transcript, complete with a fabricated session ID, when no tool call had actually run. The only reason it got caught was a standing habit of checking the target/API server directly instead of believing the model’s summary.

Why this is happening now, and why the swarm architecture is winning

The single-agent, single-context-window approach has an obvious ceiling: everything the model has learned during a long engagement has to survive inside one context, competing with everything else in that conversation. Split the work across specialist subagents with narrow system prompts and dedicated tool permissions, and give them a shared structured store instead of a shared conversation, and you get two things at once — better focus per agent, and state that survives past any one agent’s context window.

The research anchor most of these tools cite is HPTSA (Hierarchical Planning and Task-decomposition for Autonomous agents) — PentestCode’s README claims a 4.3x improvement over single-agent approaches using this design, with a strategist/coordinator agent (pentest) dispatching to thirteen specialists that all read and write one engagement-state graph: hosts, services, vulnerabilities with confidence scores, credentials, an entity-relationship graph connecting findings (EXPLOITED_VIA, PIVOT_TO, ADMIN_OF), and a cost-based Dijkstra/Yen’s-K-shortest-paths attack-path suggester running over the top of it.

The commercial end of the field backs up the architecture with results rather than benchmarks alone. XBOW’s own numbers: on a 104-scenario comparison, a seasoned human pentester took roughly 40 hours where XBOW finished in 28 minutes — about an 85x speed difference — and it became the first autonomous system recognized by Microsoft’s own MSRC bug bounty leaderboard. PentestGPT’s newer agentic pipeline, tested against XBOW’s own 104-benchmark validation suite, lands an 86.5% success rate at a median cost of $0.42 and a median time of 3.3 minutes per successful benchmark. NodeZero, in production use, routinely cracks more than half the passwords it tests in an initial Active Directory password audit — a number that says less about AI cleverness and more about how bad password hygiene still is at scale, which is its own useful finding.

What this series covers

Six more parts, in this order: a deep dive into PentestCode’s coordinator/subagent architecture and its engagement-state model (Part 2); PentestGPT’s academic lineage and what changed between its 2024 USENIX paper and its 2026 fully-autonomous pipeline (Part 3); PentAGI’s Docker-sandboxed swarm and the root/docker.sock privilege trade-off that comes with it — a genuinely different design decision from PentestCode’s root-free single-binary install (Part 4); NodeZero and XBOW paired together as the production-safety end of the spectrum, next to the open developer-tool beta status of everything else in the series (Part 5); a defender’s read on whether an agent swarm’s traffic pattern is fingerprintable, given that PentestCode’s own documentation admits it’s “not stealthy” and prone to redundant tool runs (Part 6); and, lab access permitting, a real run of PentestCode against the same Metasploitable2 target already sitting on the Proxmox lab built for AI Part 7, so this series ends with an actual transcript rather than a described one (Part 7).

I’ll keep the same rule this site has used since AI Part 7: nothing goes in a transcript that didn’t actually run, and no tool’s own summary of what it did gets trusted without checking the underlying state directly. Given what a 7B model was willing to invent when nobody was watching, that rule matters more here than almost anywhere else on this site.