AI Pentest Agents Part 7: Installing PentestCode, and Where the Sandbox Runs Out of Road
Every other part of this series has been written from documentation, source, and published results. This one is the exception — a real run of PentestCode against the same Metasploitable2 target already sitting on the isolated Proxmox lab built for AI Part 7. It didn’t happen in one step. The install checked out cleanly in the sandbox this series has otherwise been written in; the actual autonomous engagement needed a real lab, a real LLM credential, and — as it turned out — a couple of rounds of genuine debugging before it produced anything.
What checked out, in a sandbox with no root and no Docker
The sandbox has two vCPUs, 3.8 GB of RAM, no Docker, and an unprivileged account with no path to root. PentestCode’s README claim from Part 2 — self-contained binary, no Bun, Node, or runtime needed — held up exactly as written:
$ curl -fsSL https://raw.githubusercontent.com/s0ld13rr/pentestcode/main/install.sh | bash
→ Platform: linux/x64
→ Version: v0.2.3
✓ Installed pentestcode v0.2.3 to /sessions/.../.local/bin/pentestcodeNo sudo, no package manager, no Docker daemon. file and ldd against the resulting binary confirmed the claim rather than just taking the installer’s word for it: a 138 MB ELF executable depending on nothing but libc, libpthread, and libdl. That’s a genuinely different install story from PentAGI (Part 4), which needs Docker and root or docker.sock access before it does anything at all. pentestcode --help also turned up a command surface considerably richer than the README’s quick-start section covers — acp, mcp server management, github/pr <number> integration, stats, session, db, and a full skills package-manager-style subcommand family (fork, diff, doctor). What the sandbox couldn’t do was get past pentestcode auth login — no LLM credential to give it, and no equivalent to the manual-function-calling workaround Part 7 of the AI series used for plain nmap, since PentestCode’s entire premise is the coordinator deciding what happens next, not a fixed sequence I could reproduce by hand and honestly attribute to “what an agent would do.”
Getting it running on the real lab wasn’t instant either
I ran this on the Kali VM from the original Part 7 build, against Metasploitable2 on the same isolated bridge (10.10.10.20 — the VM had been powered off between uses, and its static IP had reverted to DHCP; a quick fix once I knew what to check for). The first attempt used my existing Ollama instance on pve — qwen2.5:32b, the same model that reasoned honestly about a stuck exploit in Part 7 of the AI series. Two real problems surfaced before it produced anything useful.
First, /scope and /mode came back printing what looked like raw instruction text instead of confirmations — a symptom of Ollama’s default 4096-token context getting overwhelmed by PentestCode’s system prompt plus its tool schemas. Fixed with a custom Modelfile (PARAMETER num_ctx 16384) and ollama create, since Ollama’s OpenAI-compatible endpoint doesn’t reliably pass num_ctx through from the client side. Second, the model kept evicting between turns — Ollama’s default keep_alive unloads an idle model after five minutes, and reloading 20GB from disk isn’t instant. Fixed with OLLAMA_KEEP_ALIVE=-1 on the Ollama service.
Neither fix solved the real problem, which a direct benchmark exposed cleanly:
$ time curl http://192.168.11.121:11434/api/generate -d '{"model":"qwen2.5:32b","prompt":"Say hello in one sentence.","stream":false}'
{"response":"Hello!","total_duration":19385838160,"load_duration":11508799631,
"prompt_eval_count":35,"prompt_eval_duration":6721075558,
"eval_count":3,"eval_duration":1124616771}
real 19.41sThree tokens took 19.4 seconds — roughly 5 tokens/second of prefill on 35 input tokens, and about 2.7 tokens/second of generation. ollama ps confirmed the model genuinely running at 100% CPU across the host’s cores rather than stuck or crashed, and free -h showed no memory pressure at all (22Gi available, no swap) — this wasn’t a resource ceiling in the OOM sense, it was the honest cost of a 32B model with zero GPU offload. A PentestCode turn generating a few hundred tokens of reasoning plus a structured tool call, repeated across a coordinator and however many subagents it spawns, was not going to finish this engagement in a reasonable window.
The fix was switching providers, not further tuning: pentestcode auth login → Anthropic, using a fresh API key scoped and later revoked for just this engagement. pentestcode models anthropic confirmed claude-sonnet-5 was in this release’s embedded model catalog, and the difference in practice was immediate — PentestCode’s own README ranking (“Claude Opus/Sonnet >> GPT-4o > local models for multi-agent coordination,” quoted in Part 2) held up exactly as advertised. The Ollama numbers above aren’t wasted effort — they’re a real, measured answer to “what does CPU-only local inference cost this kind of workload,” worth keeping on their own merits even though the actual engagement ran on Claude Sonnet 5.
The real engagement: three independent root vectors on one host
Scope set to 10.10.10.20 only, mode auto, and a single deliberately open-ended prompt — no mention of vsftpd, no hint at the known backdoor, just “penetration test this host, full recon, enumerate all services, attempt to gain a shell and escalate.” PentestCode’s own stats reported a final cost of $9.64 across 16 sessions and 266 messages — comfortably inside the low end of PentestCode’s own quoted $5–50 range, as expected for a single host with no Active Directory surface to chew through.
The tool-usage breakdown from pentestcode stats is worth a moment on its own:
bash 107 (45.3%)
state_query 52 (22.0%)
state_update 38 (16.1%)
read 9 ( 3.8%)
scope_check 6 ( 2.5%)
task_graph 5 ( 2.1%)
phase_control 5 ( 2.1%)
skill 4 ( 1.7%)
edit 3 ( 1.3%)
nmap_parse 2 ( 0.8%)
write 2 ( 0.8%)
task 1 ( 0.4%)
cred_spray 1 ( 0.4%)
report_gen 1 ( 0.4%)Part 2 quoted the README’s framing that parser tools are “mandatory… this ensures every finding hits the engagement state.” In this real run, raw bash outnumbers the dedicated nmap_parse tool by more than 50 to 1. That’s not necessarily the tool failing at its own stated discipline — a lot of that bash share is enumeration and exploitation work (banner grabs, manual FTP/SSH sessions, file harvesting) that genuinely has no dedicated parser to route through — but it’s a real gap between the “mandatory parser” framing and what a live engagement’s tool distribution actually looks like, and worth knowing before taking that claim at face value.
The result itself: root, independently, three separate ways. The coordinator found a pre-opened ingreslock bindshell on port 1524 (immediate uid=0, no exploitation needed), triggered the vsftpd 2.3.4 “smiley face” backdoor on port 21 exactly as designed (CVE-2011-2523 — the same vulnerability AI Part 7’s MetasploitMCP section targeted directly), and — on its own initiative, not prompted — harvested the msfadmin user’s private SSH key from the filesystem and discovered it was trusted directly in /root/.ssh/authorized_keys, establishing a fourth, more stable persistent root channel. The full engagement surfaced 24 vulnerabilities (9 critical, 9 high, 2 medium, 4 low) and 24 credential records (14 unique after dedup) — full /etc/shadow, two SSH private keys, a VNC root password decrypted via a well-known fixed DES key, MySQL root with no password, and confirmed default-credential superuser access to PostgreSQL and Tomcat Manager.
This is the clearest contrast with AI Part 7’s MetasploitMCP section available anywhere in this series. That post had to be told the specific CVE and module to run. This run found the same backdoor as one line in a 24-item findings list, treated it as one of four redundant paths to the same outcome, and kept going — enumerating, exploiting, and cross-checking services that post never touched.
The critic agent caught a real, verifiable error
The most concrete demonstration of PentestCode’s “critic” agent (Part 2) doing its stated job: a mid-engagement state update had recorded a “PostgreSQL COPY TO PROGRAM RCE” vector as available against the target. The critic’s validation pass flagged this as factually wrong — COPY ... TO/FROM PROGRAM was introduced in PostgreSQL 9.3 (2013), and this host fingerprints as PostgreSQL 8.3.0–8.3.7 (2008-era), which cannot run that syntax. The correction was recorded in state, propagated into the final report, and the report’s own PostgreSQL finding explicitly warns against citing that technique for this host. That’s a genuine, checkable technical correction an agent caught on its own — not a hedge or a disclaimer, an actual factual error in the engagement’s own prior state, fixed before it reached the report.
The critic also did the less flashy but equally real work of merging duplicate findings (two separate VNC entries and two separate SMB entries, both citing the same underlying evidence) and downgrading three “outdated version, multiple CVEs” claims — OpenSSH 4.7p1, BIND 9.4.2, and Apache’s mod_dav flag — from Medium to Low on the grounds that no actual exploit or PoC had been run against any of them, just a version number. That’s exactly the “classic overstated ‘outdated software’ pattern” the critic’s own note called it, and it’s a healthier instinct than treating every old version string as a finding — the same discipline the earlier NSE4 and Impacket content on this site has argued for when a version banner alone gets treated as proof of exploitability.
Honest failure logging, same discipline AI Part 7 asked for
Three exploit attempts — distccd (CVE-2004-2687), UnrealIRCd’s backdoor (CVE-2010-2075), and Samba’s usermap_script RCE (CVE-2007-2447) — ran and returned empty output. PentestCode recorded all three as attempted, not exploited, with the evidence field explicitly noting the failure was “inconclusive due to tooling/subagent issue, NOT evidence against the vuln” rather than silently upgrading or quietly dropping them. That’s precisely the distinction AI Part 7 found local models collapsing under load — a 7B model in that experiment fabricated an entire successful session outright rather than reporting a stuck exploit honestly. Here, on Claude Sonnet 5, the coordinator kept “the vulnerability is real” and “I obtained a shell this session” as separate claims, exactly as they should be.
What the finished report actually looks like
The reporter subagent’s output — a real deliverable, not a summary I wrote — runs an executive summary, a scope section, findings grouped by severity, an attack-path table with timestamps and MITRE ATT&CK IDs, a credentials table, and a prioritized remediation plan split into 0–48 hours, 30 days, and 90 days. The attack-path table alone is a genuinely useful artifact:
| # | Timestamp | Technique | Result | MITRE |
| 1 | 15:07:01 | Direct connect to pre-opened bindshell (1524) | Root shell, no auth | T1133 |
| 2 | 15:08:44 | vsftpd 2.3.4 smiley backdoor (CVE-2011-2523) | Root shell via port 6200 | T1190 |
| 6 | 15:12:44 | Harvested msfadmin id_rsa, found trusted by root | SSH key reuse → stable root | T1552.004 |One correction to Part 2 of this series worth making plainly: I’d said this build had apparently dropped the README’s promised findings.md in favor of a SQLite-only backend, based on not finding it in the project’s working directory. That was wrong, or at least incomplete. findings.md exists — a real, continuously-appended, human-readable log with per-entry timestamps — just nested under ~/.pentestcode/engagements/<engagement-name>/findings.md rather than the project root the README implies. Both the SQLite store and the flat file are real; the documentation just doesn’t say where the second one actually lives.
What this run actually cost, in every sense
$9.64 in API spend by PentestCode’s own count — though my actual Anthropic billing, checked after revoking the key, showed $9.58. A six-cent gap between a tool’s self-reported cost and the real invoice isn’t nothing to know: PentestCode’s stats figure is presumably computed from per-call token counts and published list pricing, while the actual bill reflects whatever Anthropic’s billing system does with rounding, cache-write/cache-read pricing tiers, and the exact timing of a usage snapshot. Neither number is wrong, they’re just not measuring the identical thing — worth remembering before treating any tool’s self-reported cost as the number that will actually show up on a card statement.
Either figure makes the same point: roughly $9.60, real dollars, to autonomously recon, exploit, and fully report on one host end to end — plus roughly 25 minutes of wall-clock time once running on Claude Sonnet 5, and, before that, a genuine multi-hour debugging session across context windows, keep-alive settings, a false “hung request” alarm that turned out to be an SSH session dying rather than the model, and a full round of benchmarking a CPU-only local model against a trivial three-token prompt just to get an honest number for how slow it actually was. That ratio — a few minutes of finished engagement against hours of getting the harness to a state where it could run at all — is worth sitting with. It’s the same lesson the AI series has returned to more than once: the interesting cost of “let an agent do this” was never the moment it worked. It’s everything that had to be true first.