# Long-Context Features in ProxyPilot ProxyPilot implements two research-backed features for handling long-running AI agent sessions: 1. **Context Compression** - Based on Factory.ai research 2. **Agentic Harness** - Based on Anthropic's "Building Effective Agents" blog Both are enabled by default and can be toggled via environment variables. --- ## Sources | Feature | Source | Link | |---------|--------|------| | Context Compression | Factory.ai Research | [Context Compression for Long-Running Agents](https://www.factory.ai/news/context-compression) | | Agentic Harness | Anthropic Blog | [Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) | --- ## 1. Context Compression (Factory.ai) ### What Factory.ai Found Factory.ai researched how to handle long coding sessions where conversation context exceeds model limits. Their key findings: - **Naive truncation fails** - Cutting old messages loses critical context - **Structured summarization works** - Preserving key information in a specific format maintains continuity - **Anchored approach is best** - Merging new info into existing summaries beats creating fresh ones ### How ProxyPilot Implements It When conversation exceeds 75% of the model's context window: 1. **Trigger**: Token count check detects threshold exceeded 2. **Summarize**: LLM generates structured summary of older messages 3. **Inject**: Summary replaces compressed messages in subsequent requests ### Structured Summary Format ``` ## Session Intent [High-level goal of the conversation] ## File Modifications - path/to/file1.go: Added function X, modified Y - path/to/file2.ts: Refactored component Z ## Decisions Made - Chose approach A over B because... - Confirmed that X should use pattern Y ## Next Steps 1. Implement feature X 2. Add tests for Y ## Technical Details - Key code snippets or patterns - Important variable names or function signatures ``` ### Configuration | Environment Variable | Default | Description | |---------------------|---------|-------------| | `CLIPROXY_TOKEN_AWARE_ENABLED` | `true` | Enable token-based compression | | `CLIPROXY_LLM_SUMMARY_ENABLED` | `true` | Use LLM for summaries (vs regex fallback) | | `CLIPROXY_COMPRESSION_THRESHOLD` | `0.75` | Trigger at this % of context | | `CLIPROXY_SUMMARY_MODEL` | `gemini-3-flash` | Model used for summarization | ### Summary Model Selection ProxyPilot uses a dedicated model for context summarization: - **Primary**: `gemini-3-flash` via Antigravity provider - **Fallback**: `gemini-3-flash-preview` (on model-not-found errors) - **Final fallback**: Regex-based extraction (on any LLM failure) This separation keeps summarization costs low while maintaining quality. The fallback chain ensures summarization never blocks the main request. ### Disable ```bash export CLIPROXY_LLM_SUMMARY_ENABLED=false ``` --- ## 2. Agentic Harness (Anthropic) ### What Anthropic Found Anthropic's research on long-running agents identified key failure modes: - Agents try to do everything at once and fail - Agents don't track state between turns - Agents skip verification ("the code looks correct") - Agents lose context of what was already done Their solution: **Structured harness with explicit state files and disciplined workflow.** ### How ProxyPilot Implements It ProxyPilot injects prompts based on conversation state: #### INITIALIZER Mode (First Request) Injected when agentic client sends first request: ``` You are the Initializer Agent. Your ONLY goal is to set up the project environment. 1. Create 'feature_list.json' - List ALL features with: - id, category, description, steps, passes: false, priority 2. Create 'claude-progress.txt' - Session log with: - [Initializer] Environment setup started 3. Create 'init.sh' - Dev server startup script DO NOT implement features. Only scaffold the harness files. ``` #### CODING Mode (After Harness Files Exist) Injected when `feature_list.json` or `claude-progress.txt` detected: ``` You are a Coding Agent. Follow this workflow: STEP 1: GET BEARINGS (MANDATORY) - pwd, git log, cat claude-progress.txt, cat feature_list.json STEP 2: SANITY CHECK - Verify existing features work - Fix regressions before new work STEP 3: PICK ONE TASK - Find highest-priority feature where passes: false - Announce: "I will implement: [description]" STEP 4: IMPLEMENT & VERIFY - Write code for ONE feature only - Run tests to verify it works - Do NOT mark complete without actual testing STEP 5: UPDATE STATE - Set passes: true in feature_list.json - Append to claude-progress.txt STEP 6: COMMIT - Git commit with feature ID in message ``` #### PASSIVE Mode For non-agentic clients (browsers, curl), no prompts injected. ### Client Detection Harness activates for these User-Agents: - `claude-cli` - `codex` - `droid` Or when header `X-ProxyPilot-Harness: true` is set. ### Configuration | Environment Variable | Default | Description | |---------------------|---------|-------------| | `CLIPROXY_HARNESS_ENABLED` | `true` | Enable agentic harness | | `CLIPROXY_MEMORY_DIR` | `~/.proxypilot/memory` | Session storage location | ### Disable ```bash export CLIPROXY_HARNESS_ENABLED=false ``` --- ## Implementation Details ### Files Modified | File | Change | |------|--------| | `internal/memory/summarizer_executor.go` | Added `CoreManagerExecutor` interface and `ManagerAuthAdapter` | | `internal/api/middleware/codex_prompt_budget.go` | Added `InitSummarizerWithAuthManager()` | | `sdk/cliproxy/service.go` | Added `coreManagerWrapper`, wired up compression | | `internal/api/server.go` | Registered `AgenticHarnessMiddleware()` | ### Files Created | File | Purpose | |------|---------| | `internal/memory/summarizer_executor_test.go` | 12 unit tests for adapter | | `internal/api/features_integration_test.go` | 3 integration tests | ### Architecture ``` Request Flow (Context Compression): Client Request │ ▼ ┌─────────────────────┐ │ PromptBudget │ │ Middleware │ └─────────────────────┘ │ ▼ ┌─────────────────────┐ │ Token Count Check │──── < 75% ────▶ Pass through └─────────────────────┘ │ │ > 75% ▼ ┌─────────────────────┐ │ PipelineSummarizer │ │ Executor │ └─────────────────────┘ │ ▼ ┌─────────────────────┐ │ ManagerAuthAdapter │ └─────────────────────┘ │ ▼ ┌─────────────────────┐ │ coreManagerWrapper │ └─────────────────────┘ │ ▼ ┌─────────────────────┐ │ Auth Manager │ │ (claude/codex/ │ │ gemini) │ └─────────────────────┘ ``` ``` Request Flow (Agentic Harness): Client Request │ ▼ ┌─────────────────────┐ │ Check User-Agent │──── Not agentic ────▶ Pass through └─────────────────────┘ │ │ Agentic client ▼ ┌─────────────────────┐ │ Detect Harness │ │ State │ └─────────────────────┘ │ ├── No harness files + short convo ──▶ INITIALIZER prompt │ ├── Harness files exist ──────────────▶ CODING prompt │ └── Long convo, no files ─────────────▶ PASSIVE (no injection) ``` --- ## What Was NOT Implemented During implementation, we removed features that were added by a previous session but **not from the original sources**: | Removed | Reason | |---------|--------| | Puppeteer MCP detection | Not in Factory.ai or Anthropic sources | | Browser automation reminders | Made up, not from research | The current implementation is pure to the original Factory.ai and Anthropic research. --- ## Testing ```bash # Run all tests go test ./... # Run specific feature tests go test ./internal/memory/... -v # Compression adapter go test ./internal/api/middleware/... -v -run Harness # Harness middleware go test ./internal/api/... -v -run Integration # Integration tests ``` ### Test Coverage | Test File | Tests | Coverage | |-----------|-------|----------| | `summarizer_executor_test.go` | 12 | Adapter, pipeline, edge cases | | `agentic_harness_test.go` | 10 | All modes, injection, session storage | | `features_integration_test.go` | 3 | Combined feature validation | --- ## Quick Reference ### Enable/Disable Features ```bash # Disable context compression export CLIPROXY_LLM_SUMMARY_ENABLED=false # Disable agentic harness export CLIPROXY_HARNESS_ENABLED=false # Disable both export CLIPROXY_TOKEN_AWARE_ENABLED=false export CLIPROXY_HARNESS_ENABLED=false ``` ### Key Integration Points ```go // Context Compression - sdk/cliproxy/service.go:516 middleware.InitSummarizerWithAuthManager(wrapper, providers) // Agentic Harness - internal/api/server.go:224 engine.Use(middleware.AgenticHarnessMiddleware()) ``` --- ## Changelog ### 2025-12-31 - **Enhanced**: Multi-format response parsing (OpenAI, Claude, Gemini) - **Added**: `SummaryModelFallbackExecutor` for model fallback (`gemini-3-flash` → `gemini-3-flash-preview`) - **Added**: `CLIPROXY_SUMMARY_MODEL` env var for custom summary model - **Added**: Dual output format (`summary.md` + `summary.json`) - **Added**: 30+ unit tests for response parsing and model fallback ### 2025-12-27 - **Implemented**: Factory.ai-style context compression with LLM summarization - **Implemented**: Anthropic-style agentic harness (INITIALIZER/CODING modes) - **Added**: `ManagerAuthAdapter` to bridge auth manager types - **Added**: `coreManagerWrapper` to adapt concrete types to interfaces - **Added**: `InitSummarizerWithAuthManager()` for initialization - **Added**: Session-based harness file storage - **Added**: Comprehensive test coverage (25+ tests) - **Removed**: Puppeteer MCP detection (not from original sources)