Tool I/O Capture
tool-capture keeps a lossless record of every tool call the agent makes and every output it
gets back, for audit, incident review and SIEM ingestion. It ships in every profile.
The firewall audit (.pi/tool-firewall-audit.jsonl) records decisions, and trace-ledger
(.pi/trace.jsonl) records facts (tool, target, hashes). Both are redacted and short.
trace-ledger redacts only the first 1 KiB of a target and stores 200 characters of it, so a
200 KB command is logged in bounded, linear time and still reports its full length in
targetOriginalLength (tests/trace-ledger-bound-smoke.ts).
Capture records the content: the exact arguments and the exact result, including output
the firewall blocked or the tool truncated.
What is recorded
One JSON object per line. Every record carries v, ts, seq (per process, increasing),
host, pid, kind, and for tool records session, rootSession, child, agent,
cwd, toolCallId and tool.
| kind | When | Payload |
|---|---|---|
session |
session start / shutdown | reason, argv, whether redacted mode is on |
call |
tool_execution_start, before the firewall decides |
args exactly as the model sent them; actionHash, argsHash |
result |
tool_execution_end, for every call |
result (content + details) exactly as the model saw it, isError, durationMs |
attachment |
a bash result pi truncated | content: pi's full-output file, stored whole |
gap |
after records were dropped under disk pressure | how many, and why |
Blocked calls get a call and a result (with isError: true and the firewall's reason), so
the record shows what the agent tried, not only what ran.
Joining with the other logs
toolCallIdis on capture records, trace-ledger entries and firewall audit records.actionHashis the firewall's action hash: sha256 of the key-sorted JSON of{toolName, input}. It matchesactionHashin the firewall audit, grants and feedback.argsHashis trace-ledger'sargsHash.rootSessionis the firewall's root session, so a subagent's records join its parent's.
Lossless encoding
Nothing is truncated or replaced by a placeholder:
- Strings larger than PI_KIT_CAPTURE_INLINE_BYTES (32 KiB) are stored as gzip
blobs named by the sha256 of their exact bytes, in blobs/<aa>/<sha256>.gz. The record
holds {"$blob":"sha256:…","bytes":n,"encoding":"utf8"}. Strings containing lone surrogates
(which UTF-8 cannot carry) use "encoding":"utf16le".
- Bytes (Buffer, Uint8Array) become blobs with "encoding":"bytes".
- Binary and garbage output (cat /dev/urandom), NULs, control characters and lone
surrogates are escaped by JSON, so every line stays valid JSON and parses back to the same
string.
- Values JSON cannot express are tagged: {"$bigint":"…"}, {"$number":"Infinity"},
{"$date":"…"}, {"$undefined":true}, {"$circular":true}, {"$function":"name"},
{"$error":…}, {"$map":[…]}, {"$set":[…]}.
- When pi truncates a bash result, it keeps the full output in a temp file
(details.fullOutputPath). Capture streams that file into a blob in the background and
links it with an attachment record.
A blob is skipped only for a stated reason, and the record says so:
{"$blob_skipped":"exceeds_max_blob_bytes"|"disk_pressure"|"write_error: …","bytes":n,"sha256":"…"}.
Storage layout
~/.pi/agent/pi-kit/capture/ 0700
segments/<host>-<pid>-<start>-<n>.jsonl the process's active segment (0600)
segments/<host>-<pid>-<start>-<n>.jsonl.gz rotated and compressed
blobs/<aa>/<sha256>.gz content-addressed large values
retention.jsonl every file the pruner deleted, and why
- Concurrency. Each process writes only its own segment, so concurrent sessions and subagents never interleave or tear lines, and the hot path takes no locks. Blobs are written to a temp file and renamed, so concurrent writers of the same blob are safe.
- Rotation. A segment rotates at
PI_KIT_CAPTURE_SEGMENT_BYTES(32 MiB) and at session shutdown, and is gzip-compressed in the background. At start, segments left uncompressed by dead processes (crash,kill -9) are compressed. - Budget. Compressed segments and blobs are kept under
PI_KIT_CAPTURE_MAX_BYTES(2 GiB). The filesystem must keepPI_KIT_CAPTURE_MIN_FREE_BYTES(1 GiB) free. When either limit is crossed, the oldest compressed segments and least recently used blobs are deleted down to 90% of the budget, and each deletion is logged inretention.jsonl. Active segments are never pruned. If space is still short, new blobs are skipped; if it is critically short, records are dropped and agaprecord says how many.
Secrets
Values are stored byte-exact, secrets included. That makes the log useful for incident
review, and it makes the log itself sensitive:
- the directory is 0700 and every file 0600;
- protected-paths blocks agent writes into it, in every policy;
- the firewall classifies reading it as a credential read. That is high tier, and it
feeds secret → egress detection;
- setting PI_KIT_CAPTURE* from the agent's shell counts as a safety-env override;
- every record lists the secret kinds it contains in secrets, e.g.
["github_token","private_key"], so a SIEM can alert on them without reading values.
PI_KIT_CAPTURE_REDACT=1 stores redacted values instead ([REDACTED:<kind>], and
redacted: true on the record). In that mode, full-output attachments are skipped, because
they cannot be scanned in place.
The scan reads at most 4 MiB of string data per record, and every pattern is bounded, so
200 KB of adversarial text such as a repeated x_token scans in bounded time
(tests/tool-capture-smoke.ts). A password_assignment key may carry a prefix or suffix
(DB_PASSWORD=, MY_SECRET_KEY =), and an unclosed quoted value is redacted to the end of
the line (tests/tool-capture-smoke.ts). A record with string data past the 4 MiB budget
carries unscanned: true, so a SIEM knows its secrets list covers only the first 4 MiB. In
redact mode the text past the budget is not stored: it is replaced with [REDACTED:unscanned]
and the record is marked redacted: true (tests/tool-capture-smoke.ts).
Failure behaviour
Capture never breaks the agent. Every hook is wrapped. A failure (unwritable directory, full
disk) is counted, reported once, and shown by /capture, and the tool call proceeds.
Capture retries opening the store on later calls.
Configuration
| Variable | Default | Meaning |
|---|---|---|
PI_KIT_CAPTURE |
on | 0 disables capture |
PI_KIT_CAPTURE_DIR |
<agent dir>/pi-kit/capture |
where the log lives |
PI_KIT_CAPTURE_REDACT |
off | 1 stores redacted values |
PI_KIT_CAPTURE_MAX_BYTES |
2 GiB | budget for compressed segments + blobs |
PI_KIT_CAPTURE_MIN_FREE_BYTES |
1 GiB | free space to leave on the filesystem |
PI_KIT_CAPTURE_SEGMENT_BYTES |
32 MiB | rotate a segment after this size |
PI_KIT_CAPTURE_INLINE_BYTES |
32 KiB | larger strings become blobs |
PI_KIT_CAPTURE_MAX_BLOB_BYTES |
budget / 4 | larger single values are recorded as skipped |
/capture shows the directory, active segment, usage against the budget, free space, and this
process's counts (records, blobs, pruned, dropped, errors). /capture prune applies the
budget now.
Shipping to a SIEM
Point a file collector (Vector, Fluent Bit, Filebeat, Splunk UF) at segments/*.jsonl for
live tailing, or at segments/*.jsonl.gz for batch ingestion. Each line is independent JSON.
Resolve $blob references by reading blobs/<first two hex>/<sha256>.gz and gunzipping it;
the sha256 of the result equals the name. Useful alert fields are secrets, isError,
tool, actionHash (to join the firewall's decision) and rootSession.
# every call and result of one tool call, across all segments
zcat -f ~/.pi/agent/pi-kit/capture/segments/* | jq -c 'select(.toolCallId=="<id>")'