Draft. Work in progress: an outline with working notes. The finished post will replace it.

/ ai-security, pentesting, research

Building an evidence-first AI pentesting agent

Findings, tool calls and conclusions should all resolve back to evidence. A design for AI-assisted testing that keeps humans in charge.

An AI pentesting agent that produces confident findings without evidence is worse than no agent at all: it creates work, not results. The design here starts from one rule: every claim must point back to something that was actually observed.

Principles

  • Local-first.
  • Scope and rules of engagement enforced deterministically, not by instruction.
  • Every tool runs through a governed adapter with typed approvals.
  • Evidence is append-only.
  • A human authorises testing actions.

Architecture

[TODO: diagram - planner, adapters, evidence store, report builder, approval path]

Threats to the agent itself

  • Prompt injection from target content.
  • Tool poisoning: output crafted to steer the next action.
  • Data leakage out of the engagement.
  • [TODO: how each is contained]

False positives

  • An unverified finding is a lead, not a result.
  • [TODO: the verification step and what counts as evidence]

Reporting

  • Every finding links to evidence entries; every evidence entry links to the call that produced it.
  • [TODO: example (synthetic) finding record]

Not a replacement for testers

[TODO: where the agent helps, where judgement stays human]

cd ../writing