Files
ProxyPilot/docs/long-context-features.md

10 KiB

Long-Context Features in ProxyPilot

ProxyPilot implements two research-backed features for handling long-running AI agent sessions:

  1. Context Compression - Based on Factory.ai research
  2. Agentic Harness - Based on Anthropic's "Building Effective Agents" blog

Both are enabled by default and can be toggled via environment variables.


Sources

Feature Source Link
Context Compression Factory.ai Research Context Compression for Long-Running Agents
Agentic Harness Anthropic Blog Building Effective Agents

1. Context Compression (Factory.ai)

What Factory.ai Found

Factory.ai researched how to handle long coding sessions where conversation context exceeds model limits. Their key findings:

  • Naive truncation fails - Cutting old messages loses critical context
  • Structured summarization works - Preserving key information in a specific format maintains continuity
  • Anchored approach is best - Merging new info into existing summaries beats creating fresh ones

How ProxyPilot Implements It

When conversation exceeds 75% of the model's context window:

  1. Trigger: Token count check detects threshold exceeded
  2. Summarize: LLM generates structured summary of older messages
  3. Inject: Summary replaces compressed messages in subsequent requests

Structured Summary Format

## Session Intent
[High-level goal of the conversation]

## File Modifications
- path/to/file1.go: Added function X, modified Y
- path/to/file2.ts: Refactored component Z

## Decisions Made
- Chose approach A over B because...
- Confirmed that X should use pattern Y

## Next Steps
1. Implement feature X
2. Add tests for Y

## Technical Details
- Key code snippets or patterns
- Important variable names or function signatures

Configuration

Environment Variable Default Description
CLIPROXY_TOKEN_AWARE_ENABLED true Enable token-based compression
CLIPROXY_LLM_SUMMARY_ENABLED true Use LLM for summaries (vs regex fallback)
CLIPROXY_COMPRESSION_THRESHOLD 0.75 Trigger at this % of context
CLIPROXY_SUMMARY_MODEL gemini-3-flash Model used for summarization

Summary Model Selection

ProxyPilot uses a dedicated model for context summarization:

  • Primary: gemini-3-flash via Antigravity provider
  • Fallback: gemini-3-flash-preview (on model-not-found errors)
  • Final fallback: Regex-based extraction (on any LLM failure)

This separation keeps summarization costs low while maintaining quality. The fallback chain ensures summarization never blocks the main request.

Disable

export CLIPROXY_LLM_SUMMARY_ENABLED=false

2. Agentic Harness (Anthropic)

What Anthropic Found

Anthropic's research on long-running agents identified key failure modes:

  • Agents try to do everything at once and fail
  • Agents don't track state between turns
  • Agents skip verification ("the code looks correct")
  • Agents lose context of what was already done

Their solution: Structured harness with explicit state files and disciplined workflow.

How ProxyPilot Implements It

ProxyPilot injects prompts based on conversation state:

INITIALIZER Mode (First Request)

Injected when agentic client sends first request:

You are the Initializer Agent. Your ONLY goal is to set up the project environment.

1. Create 'feature_list.json' - List ALL features with:
   - id, category, description, steps, passes: false, priority

2. Create 'claude-progress.txt' - Session log with:
   - [Initializer] Environment setup started

3. Create 'init.sh' - Dev server startup script

DO NOT implement features. Only scaffold the harness files.

CODING Mode (After Harness Files Exist)

Injected when feature_list.json or claude-progress.txt detected:

You are a Coding Agent. Follow this workflow:

STEP 1: GET BEARINGS (MANDATORY)
- pwd, git log, cat claude-progress.txt, cat feature_list.json

STEP 2: SANITY CHECK
- Verify existing features work
- Fix regressions before new work

STEP 3: PICK ONE TASK
- Find highest-priority feature where passes: false
- Announce: "I will implement: [description]"

STEP 4: IMPLEMENT & VERIFY
- Write code for ONE feature only
- Run tests to verify it works
- Do NOT mark complete without actual testing

STEP 5: UPDATE STATE
- Set passes: true in feature_list.json
- Append to claude-progress.txt

STEP 6: COMMIT
- Git commit with feature ID in message

PASSIVE Mode

For non-agentic clients (browsers, curl), no prompts injected.

Client Detection

Harness activates for these User-Agents:

  • claude-cli
  • codex
  • droid

Or when header X-ProxyPilot-Harness: true is set.

Configuration

Environment Variable Default Description
CLIPROXY_HARNESS_ENABLED true Enable agentic harness
CLIPROXY_MEMORY_DIR ~/.proxypilot/memory Session storage location

Disable

export CLIPROXY_HARNESS_ENABLED=false

Implementation Details

Files Modified

File Change
internal/memory/summarizer_executor.go Added CoreManagerExecutor interface and ManagerAuthAdapter
internal/api/middleware/codex_prompt_budget.go Added InitSummarizerWithAuthManager()
sdk/cliproxy/service.go Added coreManagerWrapper, wired up compression
internal/api/server.go Registered AgenticHarnessMiddleware()

Files Created

File Purpose
internal/memory/summarizer_executor_test.go 12 unit tests for adapter
internal/api/features_integration_test.go 3 integration tests

Architecture

Request Flow (Context Compression):

Client Request
      │
      ▼
┌─────────────────────┐
│ PromptBudget        │
│ Middleware          │
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ Token Count Check   │──── < 75% ────▶ Pass through
└─────────────────────┘
      │
      │ > 75%
      ▼
┌─────────────────────┐
│ PipelineSummarizer  │
│ Executor            │
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ ManagerAuthAdapter  │
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ coreManagerWrapper  │
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ Auth Manager        │
│ (claude/codex/      │
│  gemini)            │
└─────────────────────┘
Request Flow (Agentic Harness):

Client Request
      │
      ▼
┌─────────────────────┐
│ Check User-Agent    │──── Not agentic ────▶ Pass through
└─────────────────────┘
      │
      │ Agentic client
      ▼
┌─────────────────────┐
│ Detect Harness      │
│ State               │
└─────────────────────┘
      │
      ├── No harness files + short convo ──▶ INITIALIZER prompt
      │
      ├── Harness files exist ──────────────▶ CODING prompt
      │
      └── Long convo, no files ─────────────▶ PASSIVE (no injection)

What Was NOT Implemented

During implementation, we removed features that were added by a previous session but not from the original sources:

Removed Reason
Puppeteer MCP detection Not in Factory.ai or Anthropic sources
Browser automation reminders Made up, not from research

The current implementation is pure to the original Factory.ai and Anthropic research.


Testing

# Run all tests
go test ./...

# Run specific feature tests
go test ./internal/memory/... -v                    # Compression adapter
go test ./internal/api/middleware/... -v -run Harness  # Harness middleware
go test ./internal/api/... -v -run Integration      # Integration tests

Test Coverage

Test File Tests Coverage
summarizer_executor_test.go 12 Adapter, pipeline, edge cases
agentic_harness_test.go 10 All modes, injection, session storage
features_integration_test.go 3 Combined feature validation

Quick Reference

Enable/Disable Features

# Disable context compression
export CLIPROXY_LLM_SUMMARY_ENABLED=false

# Disable agentic harness
export CLIPROXY_HARNESS_ENABLED=false

# Disable both
export CLIPROXY_TOKEN_AWARE_ENABLED=false
export CLIPROXY_HARNESS_ENABLED=false

Key Integration Points

// Context Compression - sdk/cliproxy/service.go:516
middleware.InitSummarizerWithAuthManager(wrapper, providers)

// Agentic Harness - internal/api/server.go:224
engine.Use(middleware.AgenticHarnessMiddleware())

Changelog

2025-12-31

  • Enhanced: Multi-format response parsing (OpenAI, Claude, Gemini)
  • Added: SummaryModelFallbackExecutor for model fallback (gemini-3-flash → gemini-3-flash-preview)
  • Added: CLIPROXY_SUMMARY_MODEL env var for custom summary model
  • Added: Dual output format (summary.md + summary.json)
  • Added: 30+ unit tests for response parsing and model fallback

2025-12-27

  • Implemented: Factory.ai-style context compression with LLM summarization
  • Implemented: Anthropic-style agentic harness (INITIALIZER/CODING modes)
  • Added: ManagerAuthAdapter to bridge auth manager types
  • Added: coreManagerWrapper to adapt concrete types to interfaces
  • Added: InitSummarizerWithAuthManager() for initialization
  • Added: Session-based harness file storage
  • Added: Comprehensive test coverage (25+ tests)
  • Removed: Puppeteer MCP detection (not from original sources)