mirror of
https://github.com/yashshah9/agentbox.git
synced 2026-11-18 18:03:44 +00:00
docs(plans): add v0.1 completion record and v0.2 roadmap
Document implemented foundation scope, acceptance criteria, and detailed next-phase milestones following PRD + technical design standards.
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# agentbox — Planning Documents
|
||||
|
||||
| Document | Status | Description |
|
||||
|----------|--------|-------------|
|
||||
| [v0.1-completed-foundation.md](./v0.1-completed-foundation.md) | **Completed** | Initial foundation release |
|
||||
| [v0.2-next-phase-roadmap.md](./v0.2-next-phase-roadmap.md) | **Planned** | Next-phase improvements |
|
||||
@@ -0,0 +1,372 @@
|
||||
# agentbox v0.1 — Completed Foundation
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Version** | 0.1.0 |
|
||||
| **Status** | Completed |
|
||||
| **Owner** | Yash Shah |
|
||||
| **Target date** | August 2026 |
|
||||
| **Completed** | 2026-08-18 |
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
v0.1 establishes the **minimum viable product** for agentbox: a self-hosted HTTP service that executes untrusted Python code and returns structured stdout/stderr/exit metadata. The release ships a FastAPI API, a subprocess-based sandbox backend (explicitly **not production-safe**), a Python SDK, Docker Compose deployment with health checks, and three automated tests.
|
||||
|
||||
This milestone validates the core developer experience (`docker compose up` → `POST /v1/run` → result) before investing in gVisor isolation, warm pools, and multi-runtime support in v0.2+.
|
||||
|
||||
---
|
||||
|
||||
## 2. Goals
|
||||
|
||||
### 2.1 Primary goals
|
||||
|
||||
- Provide a working HTTP API for remote code execution suitable for local development and integration prototyping.
|
||||
- Ship a Python SDK that wraps the API with a minimal, typed client surface.
|
||||
- Enable single-node deployment via Docker Compose with an orchestrator-friendly health endpoint.
|
||||
- Establish project scaffolding: packaging, config, CLI, tests, and documentation conventions.
|
||||
|
||||
### 2.2 Non-goals (deferred to v0.2+)
|
||||
|
||||
- Production-grade process/network isolation (gVisor, cgroups, seccomp).
|
||||
- Warm sandbox pools or snapshot/restore.
|
||||
- Non-Python runtimes (Node.js, shell).
|
||||
- Default-deny egress or network policy enforcement.
|
||||
- TypeScript SDK.
|
||||
- Per-request CPU/memory resource limits API.
|
||||
- Multi-tenant auth, rate limiting, or audit logging.
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐ GET /health ┌──────────────────┐
|
||||
│ Agent / SDK │ ───────────────────────► │ FastAPI (app) │
|
||||
│ (httpx client) │ │ uvicorn ASGI │
|
||||
└────────┬────────┘ └────────┬─────────┘
|
||||
│ │
|
||||
│ POST /v1/run { code, language } │
|
||||
└──────────────────────────────────────────► │
|
||||
▼
|
||||
┌──────────────────────┐
|
||||
│ SubprocessSandbox │
|
||||
│ temp dir + python3 │
|
||||
│ timeout + cap output│
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
| Layer | Component | Technology |
|
||||
|-------|-----------|------------|
|
||||
| API | HTTP routes, request/response models | FastAPI, Pydantic v2 |
|
||||
| Server | ASGI process | uvicorn |
|
||||
| Sandbox | MVP execution backend | `subprocess` + temp directory |
|
||||
| Config | Environment-driven settings | pydantic-settings |
|
||||
| Client | HTTP wrapper | httpx |
|
||||
| Packaging | CLI + wheel | hatchling, click |
|
||||
| Deployment | Single-node stack | Docker Compose, healthcheck |
|
||||
| Tests | API + SDK smoke tests | pytest, FastAPI TestClient |
|
||||
|
||||
---
|
||||
|
||||
## 4. Deliverables
|
||||
|
||||
### 4.1 FastAPI service — `GET /health`
|
||||
|
||||
**Purpose:** Liveness/readiness probe for Docker, Kubernetes, and load balancers.
|
||||
|
||||
**Implementation:**
|
||||
|
||||
- Route: `GET /health`
|
||||
- Response (200): JSON body with `status`, `version`, and `backend` fields.
|
||||
- No authentication required in v0.1.
|
||||
- Synchronous handler; no dependency on sandbox availability.
|
||||
|
||||
**Example response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"version": "0.1.0",
|
||||
"backend": "subprocess"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4.2 FastAPI service — `POST /v1/run`
|
||||
|
||||
**Purpose:** Execute user-supplied Python code and return captured output.
|
||||
|
||||
**Request schema (`RunRequest`):**
|
||||
|
||||
| Field | Type | Required | Constraints |
|
||||
|-------|------|----------|-------------|
|
||||
| `code` | string | yes | 1–100,000 characters |
|
||||
| `language` | string | no | default `"python"`; only `"python"` supported |
|
||||
|
||||
**Response schema (`RunResponse`):**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `stdout` | string | Captured standard output (truncated) |
|
||||
| `stderr` | string | Captured standard error (truncated) |
|
||||
| `exit_code` | int | Process exit code |
|
||||
| `duration_ms` | int | Wall-clock execution time in milliseconds |
|
||||
|
||||
**Error handling:**
|
||||
|
||||
| Condition | HTTP status | Detail |
|
||||
|-----------|-------------|--------|
|
||||
| Unsupported `language` | 400 | `"Unsupported language: {language}"` |
|
||||
| Execution timeout | 408 | `"Execution timed out"` |
|
||||
| Validation failure (empty/oversized code) | 422 | Pydantic validation error |
|
||||
|
||||
---
|
||||
|
||||
### 4.3 Subprocess sandbox backend (`SubprocessSandbox`)
|
||||
|
||||
**Purpose:** MVP code runner for development and API contract validation.
|
||||
|
||||
**Behavior:**
|
||||
|
||||
1. Create a temporary directory (`agentbox-*` prefix).
|
||||
2. Write user code to `main.py` inside the temp directory.
|
||||
3. Execute `python3 main.py` via `subprocess.run` with `cwd` set to the temp directory.
|
||||
4. Capture stdout/stderr as text; enforce configurable timeout.
|
||||
5. Truncate stdout/stderr to `max_output_bytes` (default 1 MiB).
|
||||
6. Return `RunResult` dataclass with stdout, stderr, exit_code, duration_ms.
|
||||
7. Temp directory is destroyed on context exit.
|
||||
|
||||
**Configuration (via `Settings`):**
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `AGENTBOX_DEFAULT_TIMEOUT_SECONDS` | `30` | Subprocess timeout |
|
||||
| `AGENTBOX_MAX_OUTPUT_BYTES` | `1048576` | Max stdout/stderr capture per stream |
|
||||
| `AGENTBOX_SANDBOX_BACKEND` | `subprocess` | Backend selector (future backends stubbed in config) |
|
||||
|
||||
**Security posture (explicit):**
|
||||
|
||||
> **NOT production-safe.** The subprocess backend runs code in the same OS namespace as the API server. It provides temp-directory isolation and timeout/output caps only. It does **not** provide kernel-level sandboxing, network isolation, or resource cgroup limits. Documented upgrade path: gVisor `runsc` backend in v0.2 M1.
|
||||
|
||||
---
|
||||
|
||||
### 4.4 Python SDK (`AgentboxClient`)
|
||||
|
||||
**Purpose:** Thin HTTP client for agent integrations and scripting.
|
||||
|
||||
**Surface area:**
|
||||
|
||||
| Method | HTTP mapping | Returns |
|
||||
|--------|--------------|---------|
|
||||
| `health()` | `GET /health` | `dict[str, str]` |
|
||||
| `run(code, language="python")` | `POST /v1/run` | `dict[str, object]` |
|
||||
| `close()` | — | Closes underlying httpx client |
|
||||
|
||||
**Defaults:**
|
||||
|
||||
- Base URL: `http://localhost:8080`
|
||||
- HTTP timeout: 60 seconds (client-side; separate from sandbox execution timeout)
|
||||
|
||||
**Example usage:**
|
||||
|
||||
```python
|
||||
from agentbox.sdk.client import AgentboxClient
|
||||
|
||||
client = AgentboxClient("http://localhost:8080")
|
||||
print(client.health())
|
||||
print(client.run("print('hello')"))
|
||||
client.close()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4.5 CLI (`agentbox serve`)
|
||||
|
||||
**Purpose:** Start the API server without Docker.
|
||||
|
||||
**Commands:**
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `agentbox serve [--host] [--port]` | Launch uvicorn with settings from env/defaults |
|
||||
| `agentbox health` | Local smoke echo (does not call HTTP API) |
|
||||
|
||||
---
|
||||
|
||||
### 4.6 Docker Compose deployment
|
||||
|
||||
**Purpose:** One-command local deployment with health monitoring.
|
||||
|
||||
**Services:**
|
||||
|
||||
| Service | Image target | Port | Notes |
|
||||
|---------|--------------|------|-------|
|
||||
| `agentbox` | `runtime` | 8080 | Production-like API container |
|
||||
| `test` | `dev` | — | Mounts source; runs `pytest tests/ -v` |
|
||||
|
||||
**Healthcheck (agentbox service):**
|
||||
|
||||
```yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health')"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
```
|
||||
|
||||
**Environment defaults:**
|
||||
|
||||
- `AGENTBOX_LOG_LEVEL=INFO`
|
||||
- `AGENTBOX_DEFAULT_TIMEOUT_SECONDS=30`
|
||||
|
||||
---
|
||||
|
||||
### 4.7 Test suite (3 tests)
|
||||
|
||||
| # | File | Test | Validates |
|
||||
|---|------|------|-----------|
|
||||
| 1 | `tests/test_api.py` | `test_health` | `GET /health` returns 200, `status == "ok"` |
|
||||
| 2 | `tests/test_api.py` | `test_run_python` | `POST /v1/run` executes code, stdout contains expected output, `exit_code == 0` |
|
||||
| 3 | `tests/test_sdk.py` | `test_client_instantiation` | SDK client constructs with correct base URL and closes cleanly |
|
||||
|
||||
**Run commands:**
|
||||
|
||||
```bash
|
||||
pytest tests/ -v
|
||||
docker compose run --rm test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4.8 Documentation & packaging
|
||||
|
||||
- `README.md` — installation, usage, config table, known limitations.
|
||||
- `CHANGELOG.md` — v0.1.0 release notes.
|
||||
- `pyproject.toml` — hatchling build, Python ≥3.11, dev extras (pytest, httpx, ruff, mypy).
|
||||
- `CONTRIBUTING.md` — contributor guidelines.
|
||||
|
||||
---
|
||||
|
||||
## 5. Acceptance criteria
|
||||
|
||||
All criteria below were met for v0.1.0 release sign-off.
|
||||
|
||||
### 5.1 API
|
||||
|
||||
- [x] `GET /health` returns HTTP 200 with `status`, `version`, and `backend` fields.
|
||||
- [x] `POST /v1/run` accepts valid Python code and returns stdout, stderr, exit_code, duration_ms.
|
||||
- [x] Non-Python `language` values return HTTP 400.
|
||||
- [x] Execution exceeding timeout returns HTTP 408.
|
||||
- [x] OpenAPI schema auto-generated at `/docs` (FastAPI default).
|
||||
|
||||
### 5.2 Sandbox
|
||||
|
||||
- [x] Code executes in an ephemeral temp directory; directory removed after run.
|
||||
- [x] stdout/stderr truncated at configured byte limit.
|
||||
- [x] README and inline code document that subprocess backend is **not production-safe**.
|
||||
|
||||
### 5.3 SDK
|
||||
|
||||
- [x] `AgentboxClient` can call `/health` and `/v1/run` against a running server.
|
||||
- [x] Client supports explicit `close()` for connection cleanup.
|
||||
|
||||
### 5.4 Deployment
|
||||
|
||||
- [x] `docker compose up agentbox` starts API on port 8080.
|
||||
- [x] Docker healthcheck passes when service is healthy.
|
||||
- [x] `docker compose run --rm test` executes all 3 tests successfully.
|
||||
|
||||
### 5.5 Developer experience
|
||||
|
||||
- [x] `pip install -e ".[dev]"` installs package and dev dependencies.
|
||||
- [x] `agentbox serve` starts the API locally.
|
||||
- [x] `pytest tests/ -v` passes with zero failures.
|
||||
|
||||
---
|
||||
|
||||
## 6. Test plan (executed)
|
||||
|
||||
| Step | Action | Expected result |
|
||||
|------|--------|-----------------|
|
||||
| 1 | Run `pytest tests/ -v` | 3 passed |
|
||||
| 2 | Start server via `agentbox serve` | Listening on `:8080` |
|
||||
| 3 | `curl http://localhost:8080/health` | `{"status":"ok",...}` |
|
||||
| 4 | `curl -X POST .../v1/run -d '{"code":"print(45)"}'` | stdout contains `45`, exit_code 0 |
|
||||
| 5 | `docker compose up -d agentbox` | Container healthy within ~30s |
|
||||
| 6 | `docker compose run --rm test` | 3 passed |
|
||||
|
||||
---
|
||||
|
||||
## 7. Known limitations
|
||||
|
||||
| Limitation | Impact | Mitigation (v0.2+) |
|
||||
|------------|--------|---------------------|
|
||||
| Subprocess sandbox | Code runs in API process namespace; trivial escape to host | gVisor `runsc` backend (M1) |
|
||||
| Python-only runtime | Node/shell agents cannot execute natively | Node.js runtime (M4) |
|
||||
| Cold start per request | No pool; each run spawns fresh subprocess | Warm pool (M2) |
|
||||
| No filesystem persistence | Cannot snapshot workspace state | Snapshot/restore (M3) |
|
||||
| No network policy | Untrusted code can reach network | Default-deny egress (M5) |
|
||||
| No resource limits API | CPU/memory not capped per run | Resource limits API (M6) |
|
||||
| Single-node only | No horizontal scaling story | Out of scope until post-v0.2 |
|
||||
|
||||
---
|
||||
|
||||
## 8. Risks (retrospective)
|
||||
|
||||
| Risk | Severity | Outcome | Lesson |
|
||||
|------|----------|---------|--------|
|
||||
| Subprocess mistaken for secure sandbox | High | Mitigated via README warnings, `/health` backend field, inline `ponytail:` comment | Always surface backend identity in API and docs |
|
||||
| Output bomb ( huge stdout ) | Medium | Mitigated via `max_output_bytes` truncation | Byte caps required from day one |
|
||||
| Timeout edge cases ( hung subprocess ) | Medium | Mitigated via `subprocess.run(timeout=...)` | Map timeout to HTTP 408 consistently |
|
||||
| Scope creep into gVisor for v0.1 | Medium | Avoided; subprocess MVP shipped on time | Validate API/SDK contract before isolation investment |
|
||||
| Python 3.9 vs 3.11 mismatch in local venv | Low | `requires-python >= 3.11` in pyproject | Pin Python version in Dockerfile and CI |
|
||||
|
||||
---
|
||||
|
||||
## 9. Effort estimate vs actual
|
||||
|
||||
| Workstream | Estimate | Actual | Notes |
|
||||
|------------|----------|--------|-------|
|
||||
| FastAPI API + models | 2 days | ~2 days | Straightforward CRUD-style endpoint |
|
||||
| Subprocess sandbox | 1 day | ~1 day | Temp dir + timeout |
|
||||
| Python SDK | 0.5 day | ~0.5 day | Thin httpx wrapper |
|
||||
| Docker + Compose + healthcheck | 1 day | ~1 day | Multi-stage Dockerfile |
|
||||
| Tests (3) | 0.5 day | ~0.5 day | API + SDK smoke |
|
||||
| Docs + packaging | 1 day | ~1 day | README, CHANGELOG, pyproject |
|
||||
| **Total** | **~6 days** | **~6 days** | On estimate |
|
||||
|
||||
---
|
||||
|
||||
## 10. File inventory (shipped)
|
||||
|
||||
```
|
||||
src/agentbox/
|
||||
├── __init__.py # version string
|
||||
├── api/app.py # FastAPI routes
|
||||
├── sandbox/runner.py # SubprocessSandbox
|
||||
├── sdk/client.py # AgentboxClient
|
||||
├── config.py # Settings
|
||||
└── cli.py # serve / health commands
|
||||
tests/
|
||||
├── test_api.py # 2 tests
|
||||
└── test_sdk.py # 1 test
|
||||
compose.yaml # agentbox + test services
|
||||
docker/Dockerfile # runtime + dev targets
|
||||
pyproject.toml
|
||||
README.md
|
||||
CHANGELOG.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Sign-off
|
||||
|
||||
| Role | Name | Date | Status |
|
||||
|------|------|------|--------|
|
||||
| Owner | Yash Shah | 2026-08-18 | Approved |
|
||||
| Release | v0.1.0 tagged | 2026-08-18 | Shipped |
|
||||
|
||||
**Next:** [v0.2-next-phase-roadmap.md](./v0.2-next-phase-roadmap.md)
|
||||
@@ -0,0 +1,509 @@
|
||||
# agentbox v0.2 — Next Phase Roadmap
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Version target** | 0.2.x |
|
||||
| **Status** | Planned |
|
||||
| **Owner** | Yash Shah |
|
||||
| **Planning date** | August 2026 |
|
||||
| **Depends on** | [v0.1-completed-foundation.md](./v0.1-completed-foundation.md) (shipped 2026-08-18) |
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
v0.2 transforms agentbox from a **development prototype** into a **production-capable self-hosted sandbox platform**. The roadmap delivers six milestones in dependency order: kernel-level isolation (gVisor), latency reduction (warm pool), stateful workspaces (snapshot/restore), multi-runtime support (Node.js), network hardening (default-deny egress), and developer ecosystem expansion (TypeScript SDK + resource limits API).
|
||||
|
||||
Each milestone is independently shippable behind feature flags and backend selectors, allowing incremental rollout without blocking the full v0.2 release train.
|
||||
|
||||
---
|
||||
|
||||
## 2. Strategic goals
|
||||
|
||||
| Goal | Success metric |
|
||||
|------|----------------|
|
||||
| Production-grade isolation | Untrusted code cannot access host filesystem, network (by default), or other tenants |
|
||||
| Sub-second warm execution | P50 latency < 500 ms for pooled Python/Node runs (excluding user code runtime) |
|
||||
| Stateful agent workflows | Snapshot → mutate → restore cycle completes in < 2 s for 100 MB workspace |
|
||||
| Multi-runtime parity | Node.js runs achieve same API contract as Python |
|
||||
| Safe-by-default networking | Zero egress unless explicitly allowlisted |
|
||||
| Broad SDK adoption | TypeScript SDK published; feature parity with Python SDK |
|
||||
|
||||
---
|
||||
|
||||
## 3. Milestone overview
|
||||
|
||||
```
|
||||
v0.1 (shipped)
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ M1: gVisor runsc backend Est: 3 weeks Priority: P0 │
|
||||
└────────────────────────────┬────────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ M2: Warm pool pre-warmed sandboxes Est: 2 weeks Priority: P0 │
|
||||
└────────────────────────────┬────────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ M3: Snapshot/restore filesystem Est: 3 weeks Priority: P1 │
|
||||
└────────────────────────────┬────────────────────────────────────┘
|
||||
▼
|
||||
┌────────────────────┴────────────────────┐
|
||||
▼ ▼
|
||||
┌───────────────────┐ ┌───────────────────┐
|
||||
│ M4: Node.js │ │ M5: Default-deny │
|
||||
│ runtime │ │ egress │
|
||||
│ Est: 2 weeks P1 │ │ Est: 2 weeks P0 │
|
||||
└─────────┬─────────┘ └─────────┬─────────┘
|
||||
└──────────────────┬──────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ M6: TypeScript SDK + resource limits API Est: 2 weeks P1 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
| Milestone | Summary | Estimate | Priority | Depends on |
|
||||
|-----------|---------|----------|----------|------------|
|
||||
| **M1** | gVisor `runsc` backend | 3 weeks | P0 | v0.1 |
|
||||
| **M2** | Warm pool pre-warmed sandboxes | 2 weeks | P0 | M1 |
|
||||
| **M3** | Snapshot/restore filesystem | 3 weeks | P1 | M1 |
|
||||
| **M4** | Node.js runtime | 2 weeks | P1 | M1, M2 (recommended) |
|
||||
| **M5** | Default-deny egress | 2 weeks | P0 | M1 |
|
||||
| **M6** | TypeScript SDK + resource limits API | 2 weeks | P1 | M1, M4 (SDK parity) |
|
||||
|
||||
**Total estimated calendar time:** ~10–12 weeks (M4/M5 parallelized after M1; M6 follows M4).
|
||||
|
||||
---
|
||||
|
||||
## 4. Cross-cutting requirements (all milestones)
|
||||
|
||||
- **Backend abstraction:** Extend `SandboxBackend` protocol; select via `AGENTBOX_SANDBOX_BACKEND=runsc|subprocess`.
|
||||
- **Feature flags:** New capabilities gated by env vars until stable; subprocess backend remains for local dev.
|
||||
- **Observability:** Structured logs (structlog) with `run_id`, `backend`, `duration_ms`, `pool_hit` fields.
|
||||
- **Backward compatibility:** `POST /v1/run` request/response schema unchanged; new fields are additive only.
|
||||
- **Testing:** Each milestone adds ≥3 integration tests; CI runs against gVisor-capable runner (Linux + KVM).
|
||||
- **Documentation:** README, OpenAPI, and SDK docs updated per milestone.
|
||||
|
||||
---
|
||||
|
||||
## M1 — gVisor runsc backend
|
||||
|
||||
### Objective
|
||||
|
||||
Replace subprocess isolation with **gVisor `runsc`** OCI runtime so untrusted code executes in a user-space kernel with a separate mount/network namespace.
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **`RunscSandbox` backend** implementing the shared sandbox interface (`run_python`, future `run_node`).
|
||||
2. **OCI bundle builder** — construct `config.json` + rootfs for each execution (or pool slot).
|
||||
3. **Rootfs image** — minimal Python 3.11 base (`docker/Dockerfile.runsc-rootfs`).
|
||||
4. **Runtime integration** — invoke `runsc run` with flags: `--network=none` (initial), read-only rootfs, tmpfs workspace mount.
|
||||
5. **Config additions:**
|
||||
- `AGENTBOX_SANDBOX_BACKEND=runsc`
|
||||
- `AGENTBOX_RUNSC_BINARY=/usr/local/bin/runsc`
|
||||
- `AGENTBOX_ROOTFS_IMAGE=agentbox-rootfs:latest`
|
||||
6. **Migration path** — `/health` reports `backend: runsc`; subprocess remains available for dev.
|
||||
7. **CI job** — Linux runner with gVisor installed; smoke test executes `print(1)` via runsc.
|
||||
8. **Security documentation** — threat model doc comparing subprocess vs runsc boundaries.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] `POST /v1/run` with `backend=runsc` executes Python code; stdout/stderr/exit_code returned correctly.
|
||||
- [ ] Attempt to read `/etc/passwd` from sandbox fails (no host filesystem leak).
|
||||
- [ ] Attempt to bind/listen on network fails with `--network=none`.
|
||||
- [ ] Execution timeout kills runsc container within `timeout + 2s` grace.
|
||||
- [ ] `/health` returns `"backend": "runsc"` when configured.
|
||||
- [ ] Subprocess backend still works when `AGENTBOX_SANDBOX_BACKEND=subprocess`.
|
||||
- [ ] ≥5 integration tests pass in CI (happy path, timeout, exit non-zero, output cap, isolation).
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| gVisor incompatible with host kernel | Medium | High | Document supported kernel versions; CI matrix on Ubuntu 22.04/24.04 |
|
||||
| runsc startup latency (>2s cold) | High | Medium | M2 warm pool directly addresses; set expectations in v0.2.0 release notes |
|
||||
| OCI bundle complexity | Medium | Medium | Start with `runsc do` / minimal spec; iterate |
|
||||
| macOS dev machines cannot run runsc | High | Low | Subprocess fallback; document Linux requirement for production |
|
||||
| Debugging sandbox failures is hard | Medium | Medium | Capture runsc logs; expose `run_id` in error responses |
|
||||
|
||||
### Estimate
|
||||
|
||||
| Phase | Duration |
|
||||
|-------|----------|
|
||||
| Design + threat model | 2 days |
|
||||
| OCI bundle + runsc integration | 8 days |
|
||||
| Rootfs image + Docker updates | 3 days |
|
||||
| Tests + CI (gVisor runner) | 4 days |
|
||||
| Docs + review | 2 days |
|
||||
| **Total** | **~3 weeks** |
|
||||
|
||||
---
|
||||
|
||||
## M2 — Warm pool pre-warmed sandboxes
|
||||
|
||||
### Objective
|
||||
|
||||
Maintain a pool of **pre-initialized runsc containers** (or paused bundles) to eliminate cold-start latency for high-frequency agent tool calls.
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **`SandboxPool` manager** — configurable pool size, idle TTL, and replenishment worker.
|
||||
2. **Pool lifecycle:**
|
||||
- **Warm:** container booted, Python/Node runtime ready, workspace tmpfs mounted.
|
||||
- **Checkout:** assign slot to incoming `POST /v1/run` request.
|
||||
- **Checkin:** reset workspace (see M3 for snapshot-aware reset), return to pool or destroy if unhealthy.
|
||||
3. **Config additions:**
|
||||
- `AGENTBOX_POOL_SIZE=4` (default)
|
||||
- `AGENTBOX_POOL_IDLE_TTL_SECONDS=300`
|
||||
- `AGENTBOX_POOL_WARMUP_ON_START=true`
|
||||
4. **Metrics exposure** — pool size, available slots, checkout wait time (logged; optional `/metrics` stub for future Prometheus).
|
||||
5. **Graceful shutdown** — drain pool on SIGTERM before exit.
|
||||
6. **API additive field** — `pool_hit: bool` in `RunResponse` (optional, default omitted for backward compat).
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] With `POOL_SIZE=4`, four concurrent requests complete without cold-start runsc init.
|
||||
- [ ] P50 checkout latency < 100 ms when pool has available slots (measured in integration bench test).
|
||||
- [ ] Pool replenishes automatically after slot consumption.
|
||||
- [ ] Unhealthy slots (crash, timeout) are evicted and replaced.
|
||||
- [ ] Server starts cleanly with `POOL_WARMUP_ON_START=true`; `/health` OK after warmup.
|
||||
- [ ] Pool respects global concurrency cap; excess requests queue or return 503 with `Retry-After` (configurable).
|
||||
- [ ] ≥4 integration tests: pool hit, pool miss (cold fallback), eviction, concurrent checkout.
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Memory pressure from idle containers | High | High | Configurable pool size; idle TTL; document RAM = pool_size × rootfs footprint |
|
||||
| State leakage between checkouts | Medium | Critical | Mandatory workspace wipe on checkin; add regression test |
|
||||
| Pool deadlock under load | Low | High | Timeouts on checkout; circuit breaker to cold path |
|
||||
| runsc pause/resume not stable | Medium | Medium | Fall back to destroy-and-recreate if resume fails |
|
||||
|
||||
### Estimate
|
||||
|
||||
| Phase | Duration |
|
||||
|-------|----------|
|
||||
| Pool manager design | 2 days |
|
||||
| Checkout/checkin + warmup | 6 days |
|
||||
| Concurrency + eviction | 3 days |
|
||||
| Benchmark + tests | 3 days |
|
||||
| Docs | 1 day |
|
||||
| **Total** | **~2 weeks** |
|
||||
|
||||
**Depends on:** M1 (runsc backend).
|
||||
|
||||
---
|
||||
|
||||
## M3 — Snapshot/restore filesystem
|
||||
|
||||
### Objective
|
||||
|
||||
Enable agents to **capture workspace state** (files written during execution) and **restore** it into a subsequent sandbox run — supporting multi-step coding workflows without re-uploading context.
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Snapshot API:**
|
||||
- `POST /v1/snapshots` — create snapshot from active or last-used workspace; returns `snapshot_id`.
|
||||
- `GET /v1/snapshots/{id}` — metadata (size, created_at, runtime).
|
||||
- `DELETE /v1/snapshots/{id}` — reclaim storage.
|
||||
2. **Restore integration:**
|
||||
- `POST /v1/run` accepts optional `snapshot_id` to seed workspace before execution.
|
||||
- Pool checkin optionally auto-snapshots (configurable).
|
||||
3. **Storage backend** — local directory (`AGENTBOX_SNAPSHOT_DIR`) with content-addressed tar.zst archives; pluggable interface for S3 later.
|
||||
4. **Size limits** — `AGENTBOX_SNAPSHOT_MAX_BYTES` (default 500 MB); reject oversize with 413.
|
||||
5. **Garbage collection** — TTL-based cleanup (`AGENTBOX_SNAPSHOT_TTL_HOURS=24`).
|
||||
6. **Python SDK updates** — `client.create_snapshot()`, `client.run(code, snapshot_id=...)`.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] Run code that writes `output.txt` → snapshot → new run with `snapshot_id` → file contents preserved.
|
||||
- [ ] Snapshot size reported in metadata; oversize snapshot rejected.
|
||||
- [ ] Restore + run completes in < 2 s for 100 MB workspace (benchmark test, Linux CI).
|
||||
- [ ] Deleted snapshot returns 404 on restore attempt.
|
||||
- [ ] Snapshots isolated per tenant ID when auth is added (design hook; single-tenant OK for v0.2).
|
||||
- [ ] ≥5 tests: create, restore, delete, TTL expiry, size limit.
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Snapshot storage growth | High | Medium | TTL GC; max count per node; compression (zstd) |
|
||||
| Inconsistent state if snapshotted mid-run | Medium | High | Only snapshot on clean checkin; document semantics |
|
||||
| Cross-platform tar metadata issues | Low | Medium | Normalize permissions; test round-trip |
|
||||
| Slow restore negates pool benefits | Medium | Medium | Layer snapshot onto pooled slot; benchmark in CI |
|
||||
|
||||
### Estimate
|
||||
|
||||
| Phase | Duration |
|
||||
|-------|----------|
|
||||
| Storage format + API design | 3 days |
|
||||
| Snapshot create/restore impl | 8 days |
|
||||
| GC + limits | 2 days |
|
||||
| SDK + tests | 4 days |
|
||||
| Docs | 2 days |
|
||||
| **Total** | **~3 weeks** |
|
||||
|
||||
**Depends on:** M1. **Recommended after:** M2 (pool checkin is natural snapshot point).
|
||||
|
||||
---
|
||||
|
||||
## M4 — Node.js runtime
|
||||
|
||||
### Objective
|
||||
|
||||
Execute **JavaScript/TypeScript (via Node)** in the same sandbox infrastructure as Python, using an identical API contract.
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Runtime dispatch** — `POST /v1/run` with `"language": "node"` executes via `node` in runsc rootfs.
|
||||
2. **Rootfs update** — Node.js LTS (20.x) added to runsc rootfs image.
|
||||
3. **`run_node(code)` method** on sandbox backend interface.
|
||||
4. **Entry convention** — user code wrapped as inline `-e` script or temp `main.mjs` (ESM default).
|
||||
5. **Pool warmup** — Node slots pre-warmed alongside Python (config: `AGENTBOX_POOL_RUNTIMES=python,node`).
|
||||
6. **Error mapping** — syntax errors surface in `stderr` with non-zero `exit_code` (same as Python).
|
||||
7. **SDK update** — `client.run(code, language="node")`.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] `POST /v1/run` with `language: node` executes `console.log('hello')` → stdout contains `hello`.
|
||||
- [ ] Node `require()` of built-in modules works (`fs`, `path`); native addons blocked or documented as unsupported.
|
||||
- [ ] Timeout and output cap behave identically to Python runs.
|
||||
- [ ] Unsupported language still returns 400.
|
||||
- [ ] Pooled Node runs achieve same latency characteristics as Python (M2).
|
||||
- [ ] ≥4 tests: happy path, syntax error, timeout, unsupported import (if restricted).
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Node rootfs increases image size | High | Low | Multi-stage rootfs; slim Node distroless base |
|
||||
| npm/network needed for packages | Medium | Medium | Defer package install to v0.3; document no npm in v0.2 |
|
||||
| ESM vs CJS confusion | Medium | Low | Default to `.mjs` ESM; document convention |
|
||||
| Dual-runtime pool complexity | Medium | Medium | Separate sub-pools per runtime or unified with runtime tag |
|
||||
|
||||
### Estimate
|
||||
|
||||
| Phase | Duration |
|
||||
|-------|----------|
|
||||
| Rootfs + runsc Node integration | 4 days |
|
||||
| API dispatch + backend method | 2 days |
|
||||
| Pool warmup for Node | 3 days |
|
||||
| Tests + SDK | 3 days |
|
||||
| Docs | 1 day |
|
||||
| **Total** | **~2 weeks** |
|
||||
|
||||
**Depends on:** M1. **Recommended:** M2 for acceptable latency.
|
||||
|
||||
---
|
||||
|
||||
## M5 — Default-deny egress
|
||||
|
||||
### Objective
|
||||
|
||||
Block all outbound network traffic from sandboxes **by default**, with an explicit per-request or per-policy allowlist for approved destinations.
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Default policy** — runsc containers start with `--network=none` (or isolated namespace with iptables DROP).
|
||||
2. **Allowlist config:**
|
||||
- Global: `AGENTBOX_EGRESS_ALLOWLIST=api.openai.com:443,pypi.org:443`
|
||||
- Per-request override: `POST /v1/run` accepts optional `egress_allowlist: string[]` (subset of global).
|
||||
3. **DNS resolution** — controlled resolver or `/etc/hosts` injection for allowed hosts only.
|
||||
4. **Egress proxy sidecar (optional)** — lightweight userspace proxy enforcing SNI/port rules if `--network=none` is too restrictive for allowlist.
|
||||
5. **Audit logging** — log blocked egress attempts with destination (for debugging; no PII).
|
||||
6. **Documentation** — network policy guide for self-hosters.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] Default config: `curl https://example.com` inside sandbox fails.
|
||||
- [ ] With allowlist `example.com:443`, HTTPS GET to example.com succeeds.
|
||||
- [ ] Non-allowlisted host returns clear error in stderr or sandbox log.
|
||||
- [ ] Per-request allowlist cannot exceed global allowlist (security: no privilege escalation).
|
||||
- [ ] Python `urllib` and Node `fetch` behave consistently under policy.
|
||||
- [ ] ≥5 tests: block default, allow single host, block non-allowlisted, DNS leak prevention, per-request subset.
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| gVisor network stack limitations | Medium | High | Spike egress proxy approach early; fallback to iptables in namespace |
|
||||
| DNS rebinding / bypass | Medium | High | Resolve to IP at policy layer; short TTL re-resolve |
|
||||
| Allowlist UX too complex | Medium | Medium | Sensible defaults; document common AI API endpoints |
|
||||
| Breaks legitimate package installs | High | Low | Document that npm/pip install requires v0.3 egress policy |
|
||||
|
||||
### Estimate
|
||||
|
||||
| Phase | Duration |
|
||||
|-------|----------|
|
||||
| Network policy design | 2 days |
|
||||
| runsc/network namespace impl | 5 days |
|
||||
| Allowlist + per-request API | 3 days |
|
||||
| Tests + docs | 3 days |
|
||||
| **Total** | **~2 weeks** |
|
||||
|
||||
**Depends on:** M1. **Can parallelize with:** M4 after M1 complete.
|
||||
|
||||
---
|
||||
|
||||
## M6 — TypeScript SDK + resource limits API
|
||||
|
||||
### Objective
|
||||
|
||||
Ship a **TypeScript/JavaScript SDK** with feature parity to the Python client, and expose a **resource limits API** so callers can cap CPU time, memory, and (optionally) wall-clock per execution.
|
||||
|
||||
### Deliverables
|
||||
|
||||
#### TypeScript SDK
|
||||
|
||||
1. **Package** — `@agentbox/sdk` published to npm.
|
||||
2. **Client class** — `AgentboxClient` with `health()`, `run()`, `createSnapshot()` (if M3 shipped).
|
||||
3. **Types** — full TypeScript types for request/response models; exported from package.
|
||||
4. **Runtime** — works in Node 18+; fetch-based (no hard dependency on axios).
|
||||
5. **Tests** — vitest unit tests + optional integration test against local server.
|
||||
|
||||
#### Resource limits API
|
||||
|
||||
1. **Request schema extension:**
|
||||
|
||||
```json
|
||||
{
|
||||
"code": "...",
|
||||
"language": "python",
|
||||
"limits": {
|
||||
"timeout_seconds": 30,
|
||||
"memory_mb": 512,
|
||||
"cpu_ms": 10000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **Enforcement** — cgroups v2 inside runsc container:
|
||||
- `memory.max` from `memory_mb`
|
||||
- `cpu.max` or `cpu.max.burst` from `cpu_ms` / `timeout_seconds`
|
||||
3. **Response extension** — `limits_applied: { memory_mb, timeout_seconds }` and `oom_killed: bool`.
|
||||
4. **Defaults** — server-level defaults via env; per-request overrides capped by server max (prevent escalation).
|
||||
5. **Config:**
|
||||
- `AGENTBOX_MAX_MEMORY_MB=1024`
|
||||
- `AGENTBOX_MAX_TIMEOUT_SECONDS=120`
|
||||
6. **Python SDK update** — `run(code, limits={...})` parameter.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] `@agentbox/sdk` installs via npm; `client.run('print(1)')` works against v0.2 server.
|
||||
- [ ] TypeScript types match OpenAPI schema (generated or manually verified).
|
||||
- [ ] `limits.memory_mb=64` causes OOM kill on `bytearray(10**9)` allocation; `oom_killed: true` in response.
|
||||
- [ ] Per-request limits cannot exceed server-configured maximums (returns 400).
|
||||
- [ ] Both Python and TypeScript SDKs pass same integration test vectors.
|
||||
- [ ] ≥6 tests: SDK instantiation, limits happy path, OOM, timeout override rejected, max cap enforcement, types compile.
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| cgroups not available in all deploy envs | Medium | High | Detect at startup; degrade gracefully with warning |
|
||||
| Memory limit false positives | Low | Medium | Integration tests with known allocations |
|
||||
| npm package naming conflict | Low | Low | Scoped package `@agentbox/sdk` |
|
||||
| OpenAPI drift between SDKs | Medium | Medium | Generate types from OpenAPI spec in CI |
|
||||
|
||||
### Estimate
|
||||
|
||||
| Phase | Duration |
|
||||
|-------|----------|
|
||||
| Resource limits API + cgroups | 5 days |
|
||||
| TypeScript SDK | 4 days |
|
||||
| Python SDK limits param | 1 day |
|
||||
| Cross-SDK tests + npm publish | 3 days |
|
||||
| Docs | 1 day |
|
||||
| **Total** | **~2 weeks** |
|
||||
|
||||
**Depends on:** M1 (cgroups in runsc). **SDK parity:** M4 (Node runtime) should ship before or with M6.
|
||||
|
||||
---
|
||||
|
||||
## 5. Release plan
|
||||
|
||||
| Release | Milestones | Target | Theme |
|
||||
|---------|------------|--------|-------|
|
||||
| **v0.2.0** | M1, M5 | Week 5 | Secure isolation + default-deny network |
|
||||
| **v0.2.1** | M2 | Week 7 | Performance (warm pool) |
|
||||
| **v0.2.2** | M3 | Week 10 | Stateful workspaces |
|
||||
| **v0.2.3** | M4, M6 | Week 12 | Multi-runtime + SDK ecosystem |
|
||||
|
||||
Releases are cut when all acceptance criteria for included milestones pass CI on a gVisor-enabled Linux runner.
|
||||
|
||||
---
|
||||
|
||||
## 6. Infrastructure prerequisites
|
||||
|
||||
| Requirement | Milestones | Notes |
|
||||
|-------------|------------|-------|
|
||||
| Linux host with KVM | M1+ | gVisor requires compatible kernel |
|
||||
| gVisor `runsc` installed | M1+ | Pin version in Dockerfile |
|
||||
| cgroups v2 enabled | M6 | Default on modern Ubuntu |
|
||||
| CI runner with nested virt | M1+ | GitHub Actions `ubuntu-22.04` + gVisor install step |
|
||||
| Minimum 8 GB RAM (dev) | M2+ | Pool size × ~512 MB per slot |
|
||||
|
||||
---
|
||||
|
||||
## 7. Program-level risks
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Milestone dependency slip (M1 delays all) | Medium | High | Time-box M1 spike; subprocess fallback always available |
|
||||
| gVisor ops burden for self-hosters | Medium | Medium | Document install script; provide reference Dockerfile |
|
||||
| Scope creep (auth, multi-tenancy) | High | Medium | Explicit out-of-scope list; defer to v0.3 |
|
||||
| Insufficient CI coverage for isolation | Medium | Critical | Mandatory isolation test suite; no release without green CI |
|
||||
| Resource limits differ across kernels | Medium | Medium | Document tested matrix; feature-detect cgroups |
|
||||
|
||||
---
|
||||
|
||||
## 8. Out of scope (v0.3+)
|
||||
|
||||
- Multi-tenant authentication and API keys
|
||||
- Horizontal scaling / multi-node pool federation
|
||||
- Firecracker microVM backend
|
||||
- Shell/bash runtime
|
||||
- npm/pip package installation inside sandbox
|
||||
- Prometheus `/metrics` endpoint (stub only in M2)
|
||||
- WebSocket streaming stdout
|
||||
- GPU passthrough
|
||||
|
||||
---
|
||||
|
||||
## 9. Total effort summary
|
||||
|
||||
| Milestone | Estimate | Owner |
|
||||
|-----------|----------|-------|
|
||||
| M1 — gVisor runsc backend | 3 weeks | Yash Shah |
|
||||
| M2 — Warm pool | 2 weeks | Yash Shah |
|
||||
| M3 — Snapshot/restore | 3 weeks | Yash Shah |
|
||||
| M4 — Node.js runtime | 2 weeks | Yash Shah |
|
||||
| M5 — Default-deny egress | 2 weeks | Yash Shah |
|
||||
| M6 — TypeScript SDK + limits | 2 weeks | Yash Shah |
|
||||
| **Sequential total** | **~14 weeks** | |
|
||||
| **With M4∥M5 parallelization** | **~10–12 weeks** | |
|
||||
|
||||
---
|
||||
|
||||
## 10. Definition of done (v0.2 overall)
|
||||
|
||||
- [ ] All six milestones meet acceptance criteria.
|
||||
- [ ] Subprocess backend documented as dev-only; runsc is production recommendation.
|
||||
- [ ] README updated with gVisor install, pool config, snapshot API, egress policy, SDK examples (Python + TypeScript).
|
||||
- [ ] CHANGELOG entries for v0.2.0 through v0.2.3.
|
||||
- [ ] CI green on gVisor runner for full integration suite (≥25 tests cumulative).
|
||||
- [ ] Threat model reviewed and published in `docs/threat-model.md`.
|
||||
|
||||
---
|
||||
|
||||
## 11. References
|
||||
|
||||
- [v0.1 completed foundation](./v0.1-completed-foundation.md)
|
||||
- [gVisor documentation](https://gvisor.dev/docs/)
|
||||
- [OCI runtime spec](https://github.com/opencontainers/runtime-spec)
|
||||
- [cgroups v2](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html)
|
||||
|
||||
---
|
||||
|
||||
**Owner:** Yash Shah · **Last updated:** August 2026
|
||||
Reference in New Issue
Block a user