pi-resilient-retry

Let Pi try its native recovery first, then continue the same task if the model request still fails.

This is a protocol-independent AgentSession fallback, not a new model/provider entrance. Keep selecting your existing models in Pi or Percho; their protocol, auth and streaming remain owned by the SDK. Requires the public extension API in @earendil-works/pi-coding-agent 1.4.x.

English | 中文

Install them together

This extension keeps the task running. If you also want to resume a failed turn by hand, install pi-retry alongside it:

Their triggers differ. Stop automatic recovery before invoking manual /retry. The companion's behavior and combined-package integration have not been re-verified here.

The problem

Pi already retries transient provider errors — overloaded, 429, 5xx, timeouts, dropped streams — up to retry.maxRetries. Everything else ends the run immediately. That is the usual reason a flaky routing proxy kills a long task: a 400 from the upstream, an unmapped message like upstream sent an error event, or a relay-specific rejection.

This extension hooks agent_before_settle, the last boundary in Pi that can force one more model request. When a run ends on a retryable error and the user did not abort, it starts a native continuation, then waits out the backoff before sending the request.

403: {"code":"SUBSCRIPTION_NOT_FOUND","message":"No active subscription found for this group"} is covered by the default fallback policy as an upstream failure; retryQuotaErrors is not required. It uses the normal backoff and retry/repeat limits and does not trigger history repair.

Install from npm

Package: pi-resilient-retry. Use Pi's package manager so it loads the declared extension entry:

# Latest published npm version
pi install npm:pi-resilient-retry

# Pin a particular release
pi install npm:pi-resilient-retry@0.1.2

Requires Node.js 22.19+ and the compatible Pi extension API described above. Restart Pi or run /reload once idle. No new model/provider/auth configuration is needed, provided the host loads this extension into that session. Npm installation does not remove the Percho child-session limitation below.

Update an npm installation

# Update this installed package (when its npm source is not pinned)
pi update npm:pi-resilient-retry

# Or update all installed packages
pi update --extensions

Then restart or /reload once idle. A source pinned to @0.1.0 stays on that version; install a newer explicit version to change the pin. A Git commit/push is not an npm release, and a Pi CLI upgrade does not automatically update this package.

Git, local development and migration

pi install git:github.com/ZhiAo1120/pi-resilient-retry
pi install /path/to/pi-resilient-retry

Keep one installation source for this extension. If switching from Git/local to npm, inspect pi list, remove the old source, then install npm; do not load both copies:

pi remove git:github.com/ZhiAo1120/pi-resilient-retry
pi install npm:pi-resilient-retry

For a local installation, remove the actual path shown by pi list instead of the Git source.

Try working-tree changes before an npm release

Changes in this checkout do not update an already installed npm copy. To test the local package, remove the npm source, install this directory, then restart the host or /reload once idle:

pi remove npm:pi-resilient-retry
pi install /path/to/pi-resilient-retry

Use the same agent-directory/configuration scope as your host and keep only one source. This does not publish a release or change other installations.

Settings

All optional.

{
  "resilientRetry": {
    "enabled": true,
    "maxRetries": 0,
    "baseDelayMs": 2000,
    "maxDelayMs": 60000,
    "repeatLimit": 30,
    "retryQuotaErrors": false,
    "repairHistory": true
  }
}

maxRetries and repeatLimit reset after success, an aborted settlement, or session initialization. They bound consecutive terminal failures, not the whole session. Nonnegative finite numbers are accepted; invalid numeric values use defaults. Fractional delays are retained (subject to timer resolution). Legacy visibleRetry and noticeEvery settings are ignored and can be removed; leaving visibleRetry: true no longer adds conversation markers.

How a retry works

A retry is the system re-invoking the model, not the user speaking. The extension returns a boundary continuation that drops the failed assistant turn from context and re-issues the request. No message is ever sent on your behalf:

session:  user "hi" → assistant (error) → [context omit] → assistant (error) → [context omit] → assistant "answer"
requests: original messages and history; ordinary retry payloads stay identical

context_edit omits the failed attempt from future model context without adding a prompt or losing earlier memory. The original error and the edit remain in the append-only session file: this does not delete raw history or make the file stop growing. Pending boundary edits from earlier extensions are preserved; the plugin appends its own failed-attempt edit.

No retry messages

No user, assistant, or custom_message is injected, and the model is not asked to announce retries. Old custom messages with customType: "resilient-retry" are excluded by identity when building model input, without rewriting the session file. Real user messages, assistant responses, and tool results are preserved even when they mention pi-resilient-retry. Notices that very old versions wrote as real user messages are not guessed at and deleted by text.

Why a retry is not just the same request again

A relay answers each attempt with whichever upstream node it picks. Replaying the exact same payload can therefore fail forever: history items minted by node A carry ids that node B refuses to resolve (The requested item was created under a different Azure OpenAI resource).

Repair runs only when the upstream explicitly reports that the session ids are unusable:

Recovery is staged, still subject to the normal retry/repeat budgets:

1. ID repair: drop only optional id fields of complete replayable items. Message bodies, images, tool calls/results, call_id and reasoning remain intact. Ordinary missing-ID errors still arm repair for only one retry request.
2. Freeze a resource-mismatch boundary: before repairing a resource/account mismatch, capture fingerprints of the failed request's complete input prefix. Later requests for the same selected provider/API/model suppress only optional IDs in that old prefix. Newly added state never expands the boundary. Fingerprints stay in memory, not a second stored transcript.
3. One bounded deep recovery: if a resource/account mismatch recurs after ID repair, omit only complete reasoning items, including their encrypted content, inside that frozen prefix. Visible history and tool pairing survive; old hidden reasoning context is deliberately lost.
4. Preserve new state after recovery: keep suppressing the same old prefix, while retaining new reasoning, ciphertext, message/tool IDs and even newly appended items with identical contents. Success and Stop never widen the deletion boundary.
5. Stop on another mismatch: once the bounded deep-repair request has been sent, any further resource/account mismatch stops plugin continuation rather than starting a second lossy recovery. Ordinary transient errors still follow the usual retry policy.

If compaction, edits or shortened history invalidate the original prefix, no boundary repair is performed: matching contents alone cannot establish which state is old. Other selected models are unaffected. Reload/session replacement clears the in-memory boundary, so a later mismatch after reload can establish a new one; do not repeatedly reload to bypass the guard.

Generic errors and ordinary missing-item errors do not activate deep recovery. Outside a model with a resource-recovery boundary, ordinary requests remain unchanged. SDK routing and other extensions still own their transformations. Set repairHistory: false to disable both stages. Wire model IDs, message provider/API identity, call IDs, stored signatures and the original session record are never rewritten.

Resource recovery is declined when a verifiable full-request snapshot is unavailable or when previous_response_id, conversation or item_reference supplies server-only history. Removing those references can silently lose context. Transport state injected after the payload hook, including cached WebSocket continuations, is not cleared. Unknown/incomplete items survive. The resource-aware fixture checks old-state rejection and new reasoning replay, but it is still not proof of real-vendor acceptance.

Continuity and trust

This bounds the loss; it cannot reconstruct discarded hidden reasoning or guarantee that new conclusions are equivalent to the old analysis. A successful response proves that requests can continue, not semantic correctness. For long tasks, re-check important constraints, conclusions and next steps against retained raw records, tool evidence and tests. The extension never fabricates memory or injects a claim that this re-verification has happened.

Seeing a retry in progress

Retries use Pi / Percho's native agent-working indicator, not extension UI channels that Percho does not consume:

failure → agent_end → agent_before_settle (request continuation immediately)
        → agent_start → turn_start → context (backoff wait) → provider request

• Native activity: real agent_start and turn_start events precede backoff. The host can use its usual working state; the plugin does not fake events, error cards or a result pane.
• Terminal Pi: where the host implements setWorkingMessage, text changes from “等待重试（第 N 次）” (waiting) to “正在重试（第 N 次）” (requesting).
• Cancel / finish: use the host's Stop action or close the session. An AbortSignal listener wakes backoff immediately; timers/listeners are cleaned up, including very large delays. Success, cancellation and retry limits clear the plugin's working text. Direct disposal clears it on the genuine abort signal before the SDK invalidates the context; terminal callbacks do not access that disposed context. On every low-level agent_end, plugin text is returned to the host before any SDK-native backoff (which uses a separate controller). The late settlement callback uses event data and cached run metadata, not live context getters.
• Verification limit: SDK/RPC activity and cancellation are exercised; interactive TUI, full Percho desktop animation and result-pane visuals have not been observed.

Waiting inside agent_before_settle is too late: the preceding low-level agent_end already turned off the native indicator. Blocking a start-event handler is also too early: Pi waits for extension handlers before notifying UI subscribers. Backoff therefore runs in context, after both start events; the wait itself does not modify any messages.

There is still a normal lifecycle transition between each low-level run and its continuation. The host can briefly reset or retain its native error cards. This extension neither fabricates lifecycle events nor patches Percho to hide those records. Use the debug log below for exact attempt counts and decision reasons.

Stopping an impossible loop

The repeat guard stops an unproductive loop of identical errors; it does not prove that changing errors are safe or cheap to retry. Use a finite maxRetries if you need a bounded plugin budget. Error comparison hashes the full string after normalizing whitespace and redacting labeled trace_id / request_id values. All other error details remain significant; distinct long errors sharing a prefix do not incorrectly trip the guard.

Rotating tracing IDs no longer reset the counter. Substantively different errors can still retry without limit; the same normalized error thirty-one times in a row cannot.

The continuity guard is independent: a model that has sent one bounded deep-repair request stops on another resource mismatch, even with both maxRetries: 0 and repeatLimit: 0.

Diagnosing a silent non-retry

Set a debug log to record every decision:

{ "resilientRetry": { "debugLog": "/tmp/resilient-retry.log" } }

Records contain only allowlisted event/decision labels, counters, flags and a SHA-256 errorFingerprint. Raw upstream error text, request/response bodies, credentials, headers and the configured log path are never written. Newly created files use mode 0600; existing-file permissions are not changed. SDK/host logs and raw session records are separate.

{"event":"agent_before_settle","outcome":"error","stopReason":"error","aborted":false,"decision":"retry","attempt":1,"delayMs":2000,"repair":false}
{"event":"agent_before_settle","outcome":"error","stopReason":"error","decision":"skip: policy declined"}
{"event":"history_repaired","removed":3,"removedReasoning":1,"deepRepair":true,"snapshotItems":8}
{"event":"agent_before_settle","decision":"skip: continuity guard"}
{"event":"agent_before_settle","decision":"skip: unsafe replay"}

removed counts optional IDs; removedReasoning counts whole reasoning items omitted. deepRepair identifies the stage; snapshotItems counts the frozen original prefix. No item contents or item fingerprints are logged. skip: continuity guard declines a second lossy recovery; skip: unsafe replay means no safe full-request snapshot was available.

Normally startup records session_start. If it is missing, check both loading and a writable sink in the correct settings scope: sink failures are deliberately ignored and never break a run. Remove debugLog when done.

What it does not do

• No task/tool replay by the plugin. Completed tools/results remain in context; new tool calls chosen by the model still belong to Pi. Failed partial text can be regenerated, while the real failed response remains in raw history.
• No abort override. A cancelled run is never retried. The public aborted settlement flag also resets the budget when an adapter reports its last message as error rather than aborted.
• Explicit policy only. Quota/billing errors are excluded by default. Other terminal errors are eligible subject to the repeat/finite budget guards; error text also gates ID repair.
• No global interceptor. Direct provider calls outside a loaded AgentSession, tool HTTP calls and image-generation APIs are not automatically covered.

Relationship to Pi's built-in retry

The order is request → Pi native recovery → exhausted/not applicable → plugin continuation. The plugin never edits retry or retry.provider, overrides providers, or adds an inner stream/client retry layer. Each continuation still goes through Pi's existing native policy.

Consequently resilientRetry.maxRetries is not a total wire-attempt cap: with one native retry and one plugin continuation, repeated failures can produce four HTTP attempts. In contrast, retry.maxRetries: 0 disables Pi's native retry, while resilientRetry.maxRetries: 0 means unlimited plugin continuations. Keep those budgets distinct.

Percho and SDK limits

Current verification uses Pi CLI/SDK 1.4.0 and Percho 0.6.6, whose SDK actually exports VERSION 1.4.0. A PI_PACKAGE_DIR environment override can change reported package/assets versions; package names or old research notes alone are not runtime proof.

Percho's built-in subagent loader uses noExtensions: true, injects its permission factory only, and shares the ModelRuntime, not extension lifecycle handlers. A provider callback runs before native retry decisions; its public options do not reveal child-native exhaustion or a stable logical request ID. Thus a transparent native-first child wrapper is not shipped: it would either guess, retry too early, or need prohibited private host changes.

The current SDK already supports native-first recovery when the child explicitly loads this factory: its public agent_before_settle hook is sufficient. What is missing is Percho's child loader wiring, not a new SDK retry algorithm. Keep noExtensions: true and the permission gate, then allowlist this factory for each child. Pi CLI-based subagents can also work if their child process loads the extension; parent loading alone does not establish that. No subagent_extension:result bridge, taskId fabrication, UI plugin or installation patch is used.

Development

npm install
npm test     # policy checks + real SDK / loopback HTTP lifecycle tests, no external network
npm run check

index.ts is the lifecycle adapter; retry-core.ts owns the policy, budget, waits, repair and metadata logger. No new runtime SDK/client dependency was added. The package has one explicit extension entry and includes the core file.

Tests exercise SDK/client and native exhaustion before plugin recovery, native success/Stop bypass, quota opt-in, finite/unlimited budgets, no completed-tool replay, legacy-marker identity, full-error fingerprints, timer overflow/listener cleanup and private logging. Real Responses and Anthropic HTTP fixtures cover images, thinking/text signatures, tool pairing, hook-returned payloads, headers, usage, immutable ID repair and a genuinely truncated stream. Resource-aware fixtures verify that fresh signed reasoning/ciphertext survives later prompts and tool follow-up requests, new tools execute once, subsequent resource mismatches stop, and tracing-ID rotation and repairHistory: false work. Unit tests cover identical new items, changed prefixes, model/session isolation, cancellation and reference-only safety. These are SDK/loopback observations, not real-vendor/OAuth acceptance tests.

To run the same lifecycle tests directly against Percho's bundled SDK on macOS (without opening a desktop window or modifying the installation):

(
  ROOT=$(mktemp -d)
  trap 'rm -rf "$ROOT"' EXIT
  mkdir -p "$ROOT/agent"
  env -i PATH="$PATH" HOME="$ROOT" PI_CODING_AGENT_DIR="$ROOT/agent" PI_OFFLINE=1 \
    ELECTRON_RUN_AS_NODE=1 \
    PI_RETRY_TEST_SDK=/Applications/Percho.app/Contents/Resources/app.asar/node_modules/@earendil-works/pi-coding-agent/dist/index.js \
    /Applications/Percho.app/Contents/MacOS/Percho --experimental-strip-types --test test/*.test.mjs
)

Workspace-only packaging proofs/reports live in .local/agent-work/pi-resilient-retry-optimization/package-proof/ (not in the npm archive). They use the real CLI RPC entrance and both SDKs' normal package/JITI loader, with temporary HOME/config/session, fake keys and loopback only. Cold load, idle reload, disable/re-enable, parallel-session isolation, parent disposal, missing-factory controls and Stop are distinguished from complete host/desktop verification. No real credentials/configuration were changed; no upstream OAuth, WebSocket or native Percho child UI has been verified.

License

MIT