@tduyng/pi-context

Five things for the context window: see what fills it, retire old tool output once the window is busy, compact from the warm cache instead of a cold one, retire a finished turn on demand, and read back what compaction dropped.

When the window fills, pi compacts and a summariser reads the conversation again, from cold. That request costs the conversation it summarises. /context shows where the window went, and the other four make a compaction cheaper, rarer or smaller.

history is deferred at 134 tokens, and it arms itself for the rest of a session once a compaction has folded turns away, which is the state in which the raw turns need recovering. A session that never compacts pays nothing. A seven-turn session over RPC, which compacted twice, had the tool in its list afterwards and found a line from a read the compaction had removed. /context, /collapse and /context-cache are commands, so they cost nothing until typed. The dedupe is an event hook and adds no schema.

Where the window went

/context opens a TUI panel with a colored pixel map and a category list, and prints the same rows as text in a print or RPC session, which is where an autonomous run checks where its window went. Every consumer gets a row: system prompt, tools, user messages, assistant text, tool results, bash executions, compacted history, extension entries, free space. Categories break down by tool or custom type, sorted by token weight, and children are capped at 30 shown with a +N more line carrying their combined count.

The panel fits the window it opens in. Its body takes at most 70% of the rows, each line is truncated to the column width instead of wrapping, and a list too tall for the window keeps every category it can before spending the rest on child rows. The map is hidden below six rows or 86 columns.

Retiring old tool output and thinking

A tool result stays in the context and is billed again on every later request, and so does the thinking of every earlier turn: providers are sent it back as reasoning, and it is a quarter of the recorded transcript. Once a turn leaves the context above 150k tokens, a turn_end handler retires both. The largest old results become one line naming the call, the tool, its size, and the opening of what it said: [retired: bash \seq 1 2000`, 8893 chars, 2000 lines...]. The command comes from the tool call that the result answered. A result from read` is taken last, since the edit after it usually works against that file: 7 of the 26 reads retired in the recorded corpus were read again. Old thinking goes while the text and the tool calls beside it stay, and a message that held nothing but thinking keeps a marker so the roles still alternate. A signed thinking block goes the same way, dropped whole rather than rewritten: the signature covers the block it was streamed with, so rewriting the text leaves a signature that no longer matches, while removing the block leaves nothing to check. A live Anthropic session retired 14 messages of signed thinking in one batch and the next request went through, its context falling from 103k to 19k. The newest three turns keep their thinking, which is the assistant message a tool loop continues from.

The newest three turns are never touched, a failed result is never retired, and a target is retired once. A turn here is one assistant message and the tools it called, which is pi's own unit, so an autonomous run with a single prompt still retires. Counting prompts instead blocked retirement in exactly those runs, where the context grows most: with a forced threshold, a live ten-turn run with one prompt fired seven times and took the next request from 34,861 tokens to 16,997.

A call's arguments go the same way once the call is older than the keep window: a write carries a whole file in its arguments and answers with "ok", so tying the two together would have kept the largest arguments forever. 269,000 tokens of arguments sat live in one recorded session, all of them older than the window. Replayed through the planner over that session's real entries, 527 of 2,202 drafts carry the replacement at a 200k front, and benchmarks/retirement-replay.test.ts keeps that measurable on the frozen fixture: 12 of 31 sessions fire there, for 899 drafts and 152 stripped argument sets. /context on that session while it ran put the front at 471k of a 1M window: 119k tool-call arguments, 108k thinking, 102k tool results, 21k assistant text, and 132 results already retired. The arguments were the largest single line on a session that predates this rule. pi applies these as context_edit entries, so the raw transcripts stay in the session file and /tree walks back to them. The replacement rewrites the cached prefix from the first edited result, which is why it fires in one batch that aims at 60% of the threshold rather than a result at a time. That rewrite is billed once at full price, so a batch that would reclaim less than 15% of the context is declined and waits for the next turn. A result is eligible from 100 tokens, where its one-line notice still gives back two thirds of it. That floor was 400, and the sweep says it was the wrong one: over the recorded sessions the replay saves 36% of front tokens at 400, 43% at 200, 44% at 100, and only half a point more at 50. Measured on a priced provider, a batch fired at 64,706 tokens of context, retired eight results and left 16,187. The request after it wrote 8,123 tokens to the cache and read 7,969, where the request before had read 58,555, so the prefix that had to be rewritten was the kept one, not the retired one. At Haiku's prices that write pays back in about two later requests. The ratio is what decides it: that batch retired three quarters of the context, and what a batch writes is what it keeps, so the floor matters most where little is old enough to retire. A batch reclaiming 40% of the context keeps a prefix five times the size of what it saved, and pays back in roughly twenty requests instead. Measured on the longest session, 527 retirements across 2,520 requests cost 12 requests with a cold prefix, and every other request read a warm one. PI_CONTEXT_RETIRE_AT sets the threshold, and 0 turns it off. A batch stops firing once nothing above the floor is left: the longest recorded session had gone 40 requests without one while its context grew from 424k to 445k, because 241k of what remained was in results under the old 400-token floor. From there the front grows to the compaction threshold, which is what the threshold, not retirement, decides. How deep a batch goes is capped by the three turns kept verbatim rather than by its 60% target: replayed at 0.6, 0.45 and 0.3 the simulated total moves from -36.2% to -36.3%, with the same 12 firings. The other two knobs were swept the same way, at the 100-token floor. A batch floor of 0.10 saves what 0.15 saves with two more rewrites, and 0.20 gives half a point up to save three. The threshold is flat between 120k and 200k within a point, and 100k buys 1.2 points for four more rewrites, so 150k stays.

Measured over the recorded opencode-go sessions, the provider whose reported usage matches the transcript, 89.3% of request front tokens were spent above 120k: the same tool output, re-sent for the rest of the session. Replayed against those sessions, with each target priced from the stored text capped to what pi actually sends and scaled to the session's own reported total, the front falls from 0.87B tokens to 0.58B on results alone in nine firings. With thinking as well it falls to 0.31B, in fifteen firings, against a rewrite that costs one context each time. These are tokens sent, not money: a provider bills a cached re-read at a fraction of fresh input, often a tenth, and a rewrite at full price or more, so the bill falls by much less than the token count. What the count buys in full is a shorter context for the model to read and a compaction that comes later. Session counts from cursor are left out: its SDK reports the context it builds server side, which is 8 to 93 times what pi sends, so it prices a conversation pi does not have. Replayed at 80k, 120k, 150k and 200k, the front lands at 30%, 33%, 36% and 39% of today's, in 30, 18, 15 and 9 firings. Each step down buys a few points for a few more rewrites, and 150k is kept because it is the threshold the reader of that window can still see.

The 15% floor is what keeps a provider that inflates its usage from paying rewrites for slivers. Replayed against the recorded cursor sessions, which report 8 to 93 times the context pi sends, retirement with no floor makes the front 6.2% larger across 8 firings. With the floor those sessions land 0.2% smaller in 3 firings, and the honest providers give up 0.4 points of their 19.7% saving while dropping 3 of 13 firings.

Compacting from the warm prefix

A six-turn session over RPC compacted twice on this path, at 8 and 10 fresh input tokens each, against a transcript of 26k. The capture is the full transcript of the last request, including the leading system message that carries the prompt and the tool declarations. The summary request replays it unchanged with one instruction appended, so almost all of the input is a cache read, and the summariser sees full tool output rather than pi's 2,000-character head cuts.

The fallback is narrow on purpose. The branch is projected through pi's own buildSessionContext and the captured conversation must still be its prefix. A kept region that moved, a model change, a cold cache, a truncated reply or a provider error all fall back to pi's own compaction. Nothing is cancelled, since a compaction pi asked for always runs, on the cached path when it can and on pi's path otherwise.

That proof was too strict until it was measured: all 65 recorded compactions fell back, because the old comparison rebuilt the branch from message entries alone. It missed the summary a previous compaction injects, and it counted custom messages that a context hook, such as pi-goal's, keeps out of the request. It now compares against pi's own projection and allows a custom message the capture lacks. It allows the reverse too, since a context hook later in the chain can drop a message after the capture has seen it, and that drop read as a branch that moved. The third cause was in the walk itself: the capture drops its leading system message and the projection kept its own, so every comparison ran a user message against a system one. Replaying a real branch through the walk returned nothing at all, which is what the four "the branch changed shape" lines were.

There was a second reason it never ran. The capture was written on before_provider_request. pi's SDK emits that hook and its agent loop does not, so 1,570 of 1,570 recorded decisions had no capture to compare against. It is written on context_with_system now, the hook the loop does emit on every request, and the tests no longer fire the one that never fires.

/context-cache shows the declared lifetime, the last refresh, the warm capture, and the reason the last compaction in this process took the path it did. Each decision also lands in <agent-dir>/cached-compaction.jsonl with its reason, which is where to look when a compaction took the slow path.

A finished turn, retired

/collapse [turns] replaces finished turns with one branch summary, one turn by default. The anchor is the entry before the oldest user prompt in the span, so nothing before the task moves.

It builds the summary from the entries themselves: modified files, read-only files, command count, validation runs, the failures with the command that failed, and the last assistant text as the outcome. No model reads the span and no request goes out, which is where this differs from a compaction. It costs nothing and leaves the cache alone.

Two cases refuse. Nothing sits before the first prompt of a session, so there is no anchor, and a prompt with no assistant turn after it is work in flight rather than a finished turn.

What it drops is real: the reasoning inside a turn goes, the summary keeps one text block, and the file list caps at twelve with the remainder counted. The raw entries stay in the session file, so history recovers any of them and /tree walks back to them. The collapsed messages' usage rides along on the branch summary entry, which is where pi reads totals, so /cost still counts what those turns cost.

Reading back what compaction dropped

history searches or reads earlier conversation that left the window: pruned results, compacted spans, or earlier sessions of the project.

search walks the current branch newest first, or every session file of the project with all: true, newest file first. Multi-word queries OR-match. Echoes (compaction summaries, verbatim input records, prior history results) rank below originals, so a copy never displaces what it copied. The default limit is 10, the maximum 50, and the ranking is lexical.

read returns one entry in full, sliced to fit the remaining context. Page size is 2,000 to 20,000 characters, sized to leave room for the reply, and offset pages through. Session files are append-only JSONL, streamed line by line, so a large transcript never lands in memory.

A record of what was said

After automatic compaction, not manual /compact, a hidden custom message lists the compacted span's verbatim user inputs, ask_question answers, and visible custom messages. The cap is 20,000 characters, and the first ask, the latest ask and the latest input always survive.

A summary paraphrases a request, and the exact wording of an ask is often the requirement.

Two defects it fixes

When the agent directory and the working directory resolve to the same file through different paths, from a hardlink, a bind mount or a symlinked ancestor, pi adds the same <project_instructions> block twice. A before_agent_start handler strips the second copy by content, so two identical blocks are always redundant and blocks with different content stay.

The other is a stale read. A read puts a file's bytes in the transcript, and a later edit makes them wrong. Over the recorded sessions, 66 percent of read bytes are stale by this rule. An earlier version marked them in place, and the measurement removed it: replacing a tool result rewrites the cached prefix from that point, so on a 300k-token session one marker re-bills 200k to 300k tokens to reclaim 8 KB, and it recurs on every later edit to a path already read. Compaction rewrites the whole prefix once and pays for it once.

Three ideas that measured to nothing

Kept because the numbers are the argument against rebuilding them:

• A tool-result view selector, where a local model picked full, head, tail, signals or outline for every result over 2,000 characters. It cost 839 calls and 740k tokens and changed nothing, because not one tool-result line was ever quoted verbatim by a later message. With no label, there was nothing to score a view against.
• A compaction deferral hook. It opened at contextWindow - reserveTokens, a line a real session request never reaches, and recorded one firing in the whole history.
• A history rerank, which promoted twice in 17 searches and scored 0.612 to 0.659 AUC against a lexical floor at 0.707.

Install

pi install npm:@tduyng/pi-context

Local checkout: pi install /path/to/dotfiles/pi/agent/extensions/pi-context

Set PI_CACHE_RETENTION=long so cached compaction has a cache to read. Pi's default is 5 minutes, and a pause longer than that turns the warm path into a cold one. Nothing else is configurable.

Limits

• Cached compaction falls back to pi's own path on any change to the prefix, so it helps a long stable session most.
• /collapse drops the reasoning inside a turn, keeps one text block, and caps the file list at twelve.
• history ranks lexically. It is recall over a transcript, not a search engine.