# Long-Session Memory (ProxyPilot / CLIProxyAPI) CLIProxyAPI can keep long agentic CLI sessions running without manual `/compact` by: - Persisting trimmed/dropped history to disk (external memory) - Retrieving relevant snippets and injecting them back into the prompt when the request must be trimmed This is **not** a 7M-token context window. It is a store + retrieval loop. ## When it activates The memory flow runs inside `internal/api/middleware/codex_prompt_budget.go` for agentic CLIs: - `OpenAI Codex` - `factory-cli` / `droid` - `warp` - Stainless SDK clients (`X-Stainless-*` headers) It only injects memory when a request is oversized and must be trimmed. ## Where memory is stored Default directory: - If `util.WritablePath()` is set: `/.proxypilot/memory` - Otherwise: `./.proxypilot/memory` Per-session logs: - `.proxypilot/memory/sessions//events.jsonl` The session key is chosen in this order: 1. `X-CLIProxyAPI-Session` header (preferred) 2. `X-Session-Id` header 3. JSON `prompt_cache_key` 4. JSON `metadata.session_id` / `session_id` 5. Fallback hash of `Authorization + User-Agent` (hashed; raw values are not stored) ## Retrieval / injection - Dropped items are appended as JSONL events (`dropped_chat` / `dropped_responses`) - Retrieval is a lightweight keyword scorer over the last ~2MB of stored events - Up to 8 snippets (~6k chars) are injected by appending to the **last user message** (best-effort). ## Environment variables - `CLIPROXY_MEMORY_ENABLED`: - default: enabled - set to `0`, `false`, `off`, or `no` to disable persistence + retrieval - `CLIPROXY_MEMORY_DIR`: override the base directory used to store memory ## Persistent TODO (never dropped) CLIProxyAPI maintains an optional per-session TODO file: - `.proxypilot/memory/sessions//todo.md` If present, it is injected on **every** agentic request (before any trimming). This keeps the agent aligned even across many compression cycles. Ways to set it: - Write the file directly - Or send a request header `X-CLIProxyAPI-Todo` (the proxy stores it to `todo.md` and strips the header before forwarding upstream) Environment variables: - `CLIPROXY_TODO_ENABLED` (default: enabled) - `CLIPROXY_TODO_MAX_CHARS` (default: `4000`) ## Prompt-cache friendly scaffolding (append-only) To preserve stable prompt prefixes for long sessions, CLIProxyAPI injects session state as an **append-only scaffold** by default (it appends a new system message/input entry instead of mutating prior content). Packed block format: - `` - `` from `pinned.md` - `` from `summary.md` - `` from `todo.md` - `` when Spec Mode is enabled ## Text normalization Some clients (notably PowerShell `ConvertTo-Json`) can store TODO/pinned state with escaped sequences like `\\n` / `\\u2019`. CLIProxyAPI normalizes common escape sequences when reading/writing `todo.md`, `pinned.md`, and `summary.md` so the injected state stays readable. ## AGENTS.md pinning If `harness-root-dir` is set (or provided to the middleware), CLIProxyAPI will read `AGENTS.md` from that directory and merge it into pinned context. ## Spec Mode (optional) Spec Mode can be enabled to require a spec/plan before any code changes: - Header: `X-CLIProxyAPI-Spec-Mode: true` - Env: `CLIPROXY_SPEC_MODE=1` When enabled and not yet approved, the scaffold injects a spec prompt that asks the agent to produce a reviewable specification and wait for approval. ## Semantic memory (local embeddings) ProxyPilot can optionally embed and retrieve semantic memory using a local Ollama embed model. This runs fully local and is **always-on** when enabled. Environment variables: - `CLIPROXY_SEMANTIC_ENABLED` (default: enabled) - `CLIPROXY_SEMANTIC_MODEL` (default: `embeddinggemma`; also supports `nomic-embed-text`) - `CLIPROXY_SEMANTIC_BASE_URL` (default: `http://127.0.0.1:11434`) - `CLIPROXY_SEMANTIC_MAX_SNIPS` (default: `4`) - `CLIPROXY_SEMANTIC_MAX_CHARS` (default: `3000`) Namespace: - `X-CLIProxyAPI-Repo` / `X-Repo-Path` / `X-Workspace-Root` / `X-Project-Root` - or request `metadata.repo` - fallback: session key