Future work and known limitations
Open items, each stated as a limitation that exists today, with what would close it. Nothing here is a commitment or a schedule; see Roadmap for direction and Beta status for what the beta does and does not cover. An item is removed when the code and a test close it, not when it is written up.
Where an item implies a behaviour change in a safety mechanism (approvals, compaction, delegation), the change is reviewed as a change to that mechanism, not as a bug fix.
Verification that was not possible for this release
These are gaps in evidence, not known bugs. Each names the person or environment that can close it.
| Not verified | Why | Who can close it |
|---|---|---|
| Behaviour against real model providers: the effort prompts, the auto-mode judge's verdicts, the completion reviewer, delegation quality | The release was prepared without provider credentials; every suite uses deterministic fakes or the real pi runtime without a model | An operator with a provider, running a representative task at each effort tier |
| The autonomous runner on rootless Podman, on macOS and Windows, and in a run of hours or days | smoke:autonomy-container ran on Docker only, in a disposable sandbox; the in-container boundary probe (packages/autonomy/tests/boundary-probe.ts) is manual-only |
An operator, per Autonomous runs |
| A full autonomous run against a real model provider | The relay talks to a local fake upstream in every test | An operator with a provider key and a small budget |
| The web console's Content-Security-Policy in a browser | No browser was used | Anyone with a browser: open the console and check the progress bars render |
| The full check suite on Windows and on Node 24 | CI runs the suite on Linux with Node 26, and the pi-compat lanes with Node 22; Windows runs the per-profile install check only | CI matrix extension |
git and npm delivery for /profile and for the update path against the real public repository |
No release tag exists yet, and local mode is what the suites can exercise | The first release, then a run of /update from an install of it |
| Interactive use on pi 1.x | The suites drive pi's runtime without a terminal; pi 1.0 made the fullscreen TUI the default and that was not exercised by hand | An operator, running each profile interactively on pi 1.3 |
Delegation and recovery
- A child that fills its context window is now told why (compaction off, a failed recovery,
clamped output) and is not retried in a loop.
subagentadds a kit-authored hint only to a child that ended with no message at all (tests/subagent-nesting-smoke.ts); one that fills its window after replying still shows the child's own message, with no overflow or compaction hint. trigger-compact's mid-run resume sends a visible user message and has been tested only with fakes. It never runs inside child sessions.- Recovery itself starts only on a progress-guard escalation or a bare
/boost. A repeat verification FAIL does not enter recovery, but the verify-fail signal returns the policy'ssplitmove at E3 and above: a steer plus the workflow card, fromdecide()inpackages/extensions/src/effort/strategy.ts. A first FAIL returns no move, and a check that regresses sends nothing (tests/verify-failclosed-smoke.ts,tests/workflow-strategy-smoke.ts). The wider gap remains: no single "worse" signal (a test count, a checkpoint diff or averifier-boardverdict flip) tells recovery that an attempt made things worse, and "the same problem" has no settled definition for the signature. Whether scouts should run on a second provider for independence, whether an extension can drive/forkitself, and a token ceiling before recovery gives up are also open (Recovery orchestration mode). - Effort budgets are accounting, not enforcement. A hostile process owned by the same user can edit the ledger. Enforcing tiers at the operating-system level would need a supervisor outside the session.
Limits of the delegation kernel
Each is a limit of the delivered kernel, not a bug in a test.
- "No model too small" is a hypothesis. The kit footprint meter shows the
balancedkit tool text at 6,924 characters over 13 tools (two of them registered inactive) and the delegation tools at 3,089, under their ceilings, and counts companions from a reviewed record:pi-lensadds 18,800 characters to every request inbalanced, nearly three times the kit's own text, and is not yet reduced; pi core is unmeasured. Nothing shows that data plans raise completion on a small model, and a syntactically valid plan can still be a bad one (nothing in the test suite pins the meter). The livesmall-model/plan-valid-ratefixture records a rate per model and never reaches a model in CI; running it, plus an outcome eval per model, would close the question. - "No task too large" is unproven. Size is handled by replanning across turns inside the tier
allowances, and no test runs a large task. Delivered recursion is one nested
workflow()level plusexpandwithin one run; process depth stays 1, a deliberate retreat until grandchild cancellation is proven (tests/workflow-plan-smoke.tspins the expansion bounds, nothing pins a large task). A large-task eval and a cancellation test below the direct child would close it. - E5 "by default" is an instruction, not enforcement. Tests pin the text that tells the agent to
plan substantial work; only a live run can show a model following it (
tests/workflow-strategy-smoke.ts). A recorded live run at E5 on a substantial task would close it. - At default effort an autonomous request affords two gated items, or scouts plus one. Each
item's plan costs 4 and the engine request affords 8, so later items fall back to
subagent(tests/autonomy-selfimprove-smoke.ts). A larger engine allowance, or smaller plans that fit, would close it. - The ladder order and its thresholds are judgement.
planMinTier5,splitMinTier3,delegateScore3,reviewLines400 andstallAfter2 are unmeasured defaults in thestrategyblock ofeffort.json, and no evidence shows that workflows, the ladder or the card improve outcomes (tests/workflow-strategy-smoke.tspins the defaults and the moves). Comparing outcomes with the policy on and off, and measuring each threshold, would close it. - A budget can still be overshot. A live agent can pass a budget after its slot was granted,
input tokens are not counted, and retries count on the ledger but not in the run's
launchedtotal (tests/workflow-ledger-smoke.ts,tests/effort-smoke.ts). Whole-request accounting that includes input tokens, reconciled after each child ends, would close it. - A script's return value is not checked. A script cannot make a failed step read as
passed, but it can return a misleading value that the host has no way to judge (tests/workflow-hooks-smoke.ts). An acceptance check on the value, such as a structured output or a verifier verdict, would close it. - A revision check runs at events. A PASS or fingerprint is re-read after a tool result and at
turn_end, so an edit by another process is seen at the next event, not before the next action (tests/verify-failclosed-smoke.ts). A filesystem watch, or a check before every action, would close it. - Cancellation is proven to one level. A run's AbortController reaches its direct child; below
that,
proc.killdoes not. With process depth 1 that is the whole delivered chain, but a child given a second nesting level is not covered (tests/workflow-control-smoke.ts). Process groups, or a cancellation test through a grandchild, would close it. - Direct
subagentis not a run. It keeps its run log (.pi/subagent/<id>.log,runs.jsonl) and has no run directory orunknownstate; "a one-step run" holds in result shape only (tests/run-query-smoke.ts). A run record and journal for direct delegation would close it. - A bash-capable child can still run
pi. Named-tool tests claim named tools only: a role withbashcan start another agent process outside the ledger (tests/delegation-guard-smoke.ts). An operating-system boundary, not a tool list, would close it. - Scripts in the autonomous container are not delivered. The container runs no sandboxed script
workflow, because
bwrapis absent and an unattendedpolicy_askis denied (see Autonomous runs; no test covers it because it is not delivered). A sandbox proven inside the container and an unattended approval path for it would close it.
Known issues
protected-pathshas its own shell walker. The tool firewall'sshell.tsandprotected-paths'directWriteTargetsare two regular-expression sets, so a shell shape fixed in one must be fixed in the other (follow-up A-90 on the operator's board, outside this repository).tests/protected-paths-corpus-smoke.tscovers the shapes both understand today; a shared shell-walker module, which the extension contract today forbids, would close it.smoke:workflow-resumeflaked once in a sandbox. It failed once and passed on rerun; the cause was not established and no reproduction is recorded. Re-run it after a failure, and treat a repeat as a defect.
Approvals and the firewall
The Agentic security, alignment and autonomy proposal sets out coordinated security levels, trusted content admission, intent review, OpenAPPA integration and verified container autonomy. It includes a staged pi 0.99.2 compatibility workstream and acceptance gates. The Security level control implements the local lever; content screening, the OpenAPPA flow engine, intent ledger, detector, authenticated operator events and verified off-boundary attestation remain unimplemented. Relaxed has no additional medium-tier routine admission set. Remaining proposal work is not a current guarantee.
- Inline interpreter code is matched coarsely, not understood. The firewall and
protected-pathsshare one fail-closed rule: inline code run by a listed interpreter family, wrapper or shell that mentions an agent-configuration segment or distinctive tail next to any write- or exec-capable call is high and blocked, and it over-blocks such code by design (Security lists the families, wrappers and shells). It cannot see a path computed entirely at runtime with no distinctive segment or tail in the command text (character codes, decoded strings, a value read from a file), an unlisted interpreter or wrapper, nor code run from a script file; closing that needs the operating-system or container boundary below, not more text matching. - It is a guard rail, not a sandbox. Containing what an allowed tool can reach needs an operating-system or container boundary (the autonomous runner provides one).
- Low-tier actions run unasked in every mode, including covered core-tool reads, explicitly
allowed external APIs and plain shell network GETs. Read-like external names confer no authority
(
tests/firewall-shell-smoke.ts,tests/firewall-gate-smoke.ts). Tightening allowed reads would require a separate policy choice. - Package-manager and opaque-tool network use cannot be attributed to a host in unattended runs; the container network is the stop.
secret-guardis in two profiles only. It loads inbalancedandlong-horizon; whether the other profiles should load it needs data on its false-positive rate (its path rule blocks every write to a path containingsecretorcredentials).- Unreviewed companion shell/edit tools (for example
pi-lean-ctx) need exact policy entries or operator approval; parsing a shell-looking field cannot confer trust (tests/firewall-shell-smoke.ts). Reviewed companion helpers have shipped entries, but multi-target mutations still ask. Covering more shapes requires reviewed effect adapters. - Exact tool names do not bind implementations. Replacement code inherits the policy for its registered name, and direct I/O by in-process extensions is outside interception. No test proves isolation or implementation binding; that needs a boundary outside the pi process. See Security model.
- Nested-call evidence uses synthetic events.
tests/firewall-gate-smoke.tsmodels pi 1.3'sparentToolCallIdand child ids and proves parent approvals do not authorise children. No smoke drives the real codemode worker orctx.executeTool()through the installed pi runtime. - Unattended mode trusts the supervisor's claim that its boundary exists. It requires the contract to be on a read-only mount (or another user's unwritable file) seen from the initial user namespace, which an agent's shell cannot fake, and a forged launch from an interpreter is a safety-setting override. What remains: something that can run code inside the pi process, or act as root. Tying the contract to a secret only the supervisor holds would close the last gap, and would also let workers under a rootless engine run unattended, which the namespace rule currently refuses. The autonomy boundary probe does not yet report whether the worker is in the initial user namespace.
- Inline interpreter code that builds a credential path at run time (rather than naming it literally) is not recognised by the classifier.
- Approvals bind command text, not effect. An approved
git pushfollows a re-pointed remote, and an approved script runs after an edit. Binding an approval to the resolved effect (the remote URL, the script's content hash) would close it, at the cost of more prompts. - A script the agent wrote can set the unattended variables where the classifier sees only
bash run.sh. The provenance rule stops it from working; the classifier cannot see it. - An operator-set human-console directory is trusted as given.
PI_KIT_HUMAN_CONSOLE_DIRis the root of the request write in both the tool firewall and the engagement governance extension, and the helper trusts a root itself. If the operator points it at a path inside a repository that ships a link there, thependingandresolveddirectories and the request file are created through that link. The default.pi/human-consolepath is the checked one. Refusing a linked root would also refuse a link the operator made on purpose, so this needs a decision on what a console root may be. Nothing pins the linked-root case.
Profiles and compaction
- A profile switch cannot undo package payloads that
pi installalready fetched, a concurrent write to pi's settings, or a hard kill of pi part-way through. Rollback restores the settings files it snapshotted. The verifier may also roll back a good switch where Luna's entry cannot be identified (symlinked or unusual npm layouts); git and npm delivery of/profileare less exercised than local mode. - Small windows outside
lite. pi's default compaction reserve (16384 tokens) on a 32k window makes pi compact almost every turn.litecarries its owncompactionsettings and/compaction statuswarns elsewhere, but no other profile sets a reserve for a small model.
Remote control
- Critical-tier approval cards start on Allow once, like every other card (the SEC-5 decision, Remote control). Starting critical cards on Deny, or requiring an explicit gesture for them, would compensate further.
- The Android app lock is off by default. Turning it on by default needs a first-run flow that explains it.
Web console
- The session panel shows an unknown context percentage as 0%.
- The login link is passed to
xdg-openas an argument, and the token file is readable by the agent the console drives (both are documented in Web console; the firewall classifies reading the token file as a credential read, which asks you but is not a lock). Moving the token behind a one-time exchange, so it never appears in a command line, would close the first.
Releases and repository governance
- Signed release tags. Tags are annotated, not signed, and
/updateverifies no signature. Signingv*tags, publishing the allowed signers and verifying them in/updatewould bind a release to the operator's key. - A branch-namespace rule restricting who may create which branch names.
- A scheduled settings drift check that fails when the live repository settings differ from
.github/expected-settings.json; it is run by hand before releases. - A blocking ReDoS rule in the
semgrepjob. ESLint rejects non-literal regular expressions, and CodeQL's ReDoS queries block a merge only through the declared code-scanning merge protection; no Semgrep rule for slow patterns is added. - Image scanning in CI. Container images are scanned by hand when they change (Vulnerability management).
Dependencies
Candidates that were evaluated and deliberately not adopted for the beta (pi-readseek 0.12.x,
pi-subagents, TypeScript 7) are recorded with their reasons: TypeScript 7 and pi-readseek in
Supply chain, and pi-subagents in
Lessons from the private system. Revisit each with its own review.
Quality of long-running work
- Long-context reliability in planning and assessment. Plan quality can degrade silently in very long sessions. Open question: which of the sieve, the goal file and the task graph carries the load, and how to measure it.
- Skill disclosure and use. How often skills are actually loaded when relevant, and whether the routing hints help or duplicate machinery, has not been measured.
- Injection review. A written review of untrusted-content handling across context contributions, tool results and memory recall (an injection inventory with a test per path) is still to be done.
Token efficiency not yet done
Measured on 2,868 requests of unattended worker sessions (tool results were 62% of prompt characters; the cache hit ratio was 95.6%). The caps, carried notes, lazy run tools and the footprint meter are delivered (Efficiency and loops); these are not, and nothing in the repository pins any of them.
- pi native tool deferral. pi 1.x can register a tool as
exposure: "deferred"and ship a built-intool_search; Luna uses onlydefaultActive: falseplus its own wake-up. Using the native mechanism for the 22 of 44 tools that no measured session called (mostly pi-lens and pi-readseek tools, 15.2k characters of schema) needs a reviewed raise of the pi floor (peerDependenciesis 0.85.1 anddefaultActiveis ignored below pi 1.x), and a measurement of pi-lens and pi-readseek in a real session, which the meter cannot see. - Action-style tool families. Folding
workflow_run,workflow,workflow_statusandworkflow_control(and the memory and task-graph families) into one tool per family with anactionfield would cut schema text further but renames tools the firewall policy, the delegation guard and the effort hide list name, so it is a breaking change with its own migration. - The script
workflowtool is still active from the first request. It is how a run starts, so waking it needs a trigger of its own (standing mode, or askill_searchhint); until one exists, its 908 characters stay. - Compaction threshold.
trigger-compactfires at 100k tokens. Whether 60k preserves what a cycle needs while cutting the ~18% of prompt tokens above 60k per request is untested; it needs an A/B over several cycles, not a guess. - Repeated reads. 429 of 1,246 file reads repeated a path already read in the same session
(1.76M characters). The progress guard now steers reading to
read; stubbing a repeat read with "unchanged since your last read" is not built. - Capping pi's built-in
bashandreadbelow 50 KB. A 10k-character cap on bash results would have saved about 5% of prompt tokens in a simulation, but truncating test output can hide the failing line; it needs head, tail and error-line handling and its own test. - Provider flagging of carried notes. Whether OpenRouter's prompt-injection filter still flags
notes appended through
context(still user-role at the provider) is unmeasured; a 20-request canary per delivery option (tool-result suffix, mid-transcript system role,contextsplice) would settle it.context-sieve,orchestratorandverify-gatemessages still use the stored channel. - Manager redesign. The
orchestratorand the effort policy both steer delegation; merging them behind one decision is a design change, not an efficiency tweak.
Startup
- Extensions load before pi draws anything. Luna prints one erasable line on a cold start
(
docs/control-center.md, "Startup time"), but no loading progress or[ OK ]list: that needs a pi hook that runs before extension loading and a way to draw the TUI early. - The cold start is pi's loader. The compile cache is in the temporary directory, which a
reboot empties on a memory-backed
/tmp. A persistent cache location is a pi loader option (jitifsCache); Luna cannot set it from an extension. Precompiling does not help, because the loader transforms JavaScript as well. Dropping or lazily loadingpi-lens(3.8 s cold, 0.23 s warm) would help most, and needs that companion to register its tools later. - Extension load is sequential, so every companion adds its import time to the first frame.
Related records
- Roadmap
- Security model
- Recovery orchestration mode: the design behind
recovery-orchestrator