Files
skillshare/ai_docs/tests/mcp_runbook.md
T
Willie c4ab37a0a2 feat(mcp): support Pi built-in MCP
Support Pi 0.99.0 built-in MCP alongside existing adapter and extension modes, with mode-aware paths, imports, and per-server options.

Preserve custom native settings and remove cleared options only when explicitly requested and ownership is unchanged. Align CLI, dashboard previews, exposure controls, and localized documentation; simplify complex MCP components.

Add regression coverage for import dialects, escaped values, mode selection, option pruning, conflicts, and migration previews. Verified in the devcontainer with Go vet and tests, MCP integration, make check, UI build and 527 tests, five-language website build, and ai-context validation.
2026-09-30 13:35:33 +08:00

11 KiB

CLI E2E Runbook: Portable MCP Configuration

Scope

Verify inline and external declarations, global/project destinations, preview, idempotence, removal, import/adoption, conflict protection and entry restoration. No server is launched and all endpoints are inert example URLs.

Environment

Run inside the devcontainer with a fresh ssenv HOME. Each step creates its own config directory under the isolated HOME. Use the newly built ss binary. Do not run these steps against a real Agent configuration.

Steps

Step 1: Inline lifecycle with preview and removal

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-inline.XXXXXX")
export SKILLSHARE_CONFIG="$MCP_CASE/config.yaml"
printf 'targets: {}\n' > "$SKILLSHARE_CONFIG"
ss mcp add e2e-inline --url https://example.com/mcp --target claude -g >/dev/null
ss sync mcp --dry-run --json -g | jq -e '.changes[0].action == "add"' >/dev/null
ss sync mcp --json -g > "$MCP_CASE/applied.json"
jq -e '.applied | length == 1' "$MCP_CASE/applied.json" >/dev/null
ss sync mcp --json -g | jq -e '.applied == [] and .plan.changes[0].action == "unchanged"' >/dev/null
ss mcp remove e2e-inline --sync -g >/dev/null
ss mcp list --json -g

Expected:

  • exit_code: 0
  • jq: .changes == []
  • jq: .blocked == false

Step 2: External source edits and missing-file protection

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-external.XXXXXX")
export SKILLSHARE_CONFIG="$MCP_CASE/config.yaml"
printf 'sources:\n  mcp: ./mcp.yaml\nmcp:\n  targets: [claude]\n' > "$SKILLSHARE_CONFIG"
printf 'servers: {}\n' > "$MCP_CASE/mcp.yaml"
cp "$SKILLSHARE_CONFIG" "$MCP_CASE/config.before"
ss mcp add e2e-external --url https://example.com/external --sync -g >/dev/null
cmp "$SKILLSHARE_CONFIG" "$MCP_CASE/config.before"
cp "$HOME/.claude.json" "$MCP_CASE/native.before"
mv "$MCP_CASE/mcp.yaml" "$MCP_CASE/mcp.saved"
if ss sync mcp -g > "$MCP_CASE/error" 2>&1; then exit 1; fi
cmp "$HOME/.claude.json" "$MCP_CASE/native.before"
mv "$MCP_CASE/mcp.saved" "$MCP_CASE/mcp.yaml"
ss mcp list --json -g

Expected:

  • exit_code: 0
  • jq: .blocked == false
  • jq: .changes[0].action == "unchanged"

Step 3: Import refusal, explicit replace, save-only and later drift conflict

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-import.XXXXXX")
export SKILLSHARE_CONFIG="$MCP_CASE/config.yaml"
export CLAUDE_CONFIG_DIR="$MCP_CASE/client"
mkdir -p "$CLAUDE_CONFIG_DIR"
printf 'targets: {}\n' > "$SKILLSHARE_CONFIG"
printf '{"mcpServers":{"imported":{"command":"example-mcp","env":{"API_TOKEN":"literal-secret"}}},"personal":true}\n' > "$CLAUDE_CONFIG_DIR/.claude.json"
cp "$CLAUDE_CONFIG_DIR/.claude.json" "$MCP_CASE/before"
if ss mcp import imported --from claude --target claude -g > "$MCP_CASE/refused" 2>&1; then exit 1; fi
cmp "$CLAUDE_CONFIG_DIR/.claude.json" "$MCP_CASE/before"
ss mcp import imported --from claude --target claude --replace -g >/dev/null
cmp "$CLAUDE_CONFIG_DIR/.claude.json" "$MCP_CASE/before"
ss sync mcp -g >/dev/null
sed -i 's/example-mcp/changed-mcp/' "$CLAUDE_CONFIG_DIR/.claude.json"
if ss sync mcp -g > "$MCP_CASE/error" 2>&1; then exit 1; fi
ss mcp list --json -g

Expected:

  • exit_code: 0
  • jq: .blocked == true
  • jq: .changes[0].action == "conflict"

Step 4: Project native destinations and restore

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-project.XXXXXX")
mkdir -p "$MCP_CASE/.skillshare"
printf 'targets: []\n' > "$MCP_CASE/.skillshare/config.yaml"
cd "$MCP_CASE"
ss mcp add project-docs --url https://example.com/project --target claude --target codex --sync -p >/dev/null
test -f .mcp.json
test -f .codex/config.toml
ss mcp remove project-docs --sync --json -p > removal.json
MCP_BACKUP=$(jq -r '.backupIds[0]' removal.json)
ss mcp restore "$MCP_BACKUP" --dry-run --json -p | jq -e '.blocked == false' >/dev/null
ss mcp restore "$MCP_BACKUP" --json -p

Expected:

  • exit_code: 0
  • jq: .applied | length == 1
  • jq: .plan.changes[0].action == "restore"

Step 5: OpenCode, Grok and Antigravity project sync

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-extra-clients.XXXXXX")
mkdir -p "$MCP_CASE/.skillshare"
printf 'targets: []\n' > "$MCP_CASE/.skillshare/config.yaml"
cd "$MCP_CASE"
printf '{\n// keep\n"model":"example"\n}\n' > opencode.jsonc
ss mcp add company-docs --url https://example.com/mcp --target opencode --target grok --target antigravity --sync -p >/dev/null
test ! -f opencode.json
grep -q '// keep' opencode.jsonc
grep -q 'remote' opencode.jsonc
grep -q 'mcp_servers.company-docs' .grok/config.toml
jq -e '.mcpServers["company-docs"].serverUrl == "https://example.com/mcp"' .agents/mcp_config.json >/dev/null
ss mcp remove company-docs --sync --json -p > removal.json
MCP_BACKUP=$(jq -r '.backupIds[0]' removal.json)
ss mcp restore "$MCP_BACKUP" --json -p

Expected:

  • exit_code: 0
  • jq: .applied | length == 1
  • jq: .plan.changes[0].action == "restore"

Step 6: Agent-specific fields survive and a pulled change converges

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-converge.XXXXXX")
export SKILLSHARE_CONFIG="$MCP_CASE/config.yaml"
export CODEX_HOME="$MCP_CASE/codex"
mkdir -p "$CODEX_HOME"
printf 'targets: {}\n' > "$SKILLSHARE_CONFIG"
printf '[mcp_servers.docs]\nurl = "https://example.com/mcp"\nstartup_timeout_sec = 30\n' > "$CODEX_HOME/config.toml"
ss mcp import docs --from codex --target codex --sync -g >/dev/null
ss mcp add docs --url https://changed.example/mcp --target codex --replace --sync -g >/dev/null
grep -q 'startup_timeout_sec = 30' "$CODEX_HOME/config.toml"
sed -i 's/changed.example/pulled.example/' "$SKILLSHARE_CONFIG" "$CODEX_HOME/config.toml"
ss mcp list --json -g

Expected:

  • exit_code: 0
  • jq: .blocked == false
  • jq: .changes[0].action == "unchanged"

Step 7: Expanded project clients and Goose YAML import

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-expanded.XXXXXX")
mkdir -p "$MCP_CASE/.skillshare"
printf 'targets: []\n' > "$MCP_CASE/.skillshare/config.yaml"
cd "$MCP_CASE"
ss mcp add browser --target amp --target copilot --target factory --target gemini --target junie --target kiro --target warp --sync -p -- npx -y @playwright/mcp@latest >/dev/null
jq -e '."amp.mcpServers".browser.command == "npx"' .amp/settings.json >/dev/null
jq -e '.mcpServers.browser.type == "local" and .mcpServers.browser.tools == ["*"]' .github/mcp.json >/dev/null
ss sync mcp --json -p | jq -e '.applied == [] and (.plan.changes | length == 7)' >/dev/null
ss mcp add remote --url https://example.com/mcp --target gemini --sync -p >/dev/null
jq -e '.mcpServers.remote.httpUrl == "https://example.com/mcp" and .mcpServers.remote.url == null' .gemini/settings.json >/dev/null
printf 'extensions:\n  imported:\n    type: stdio\n    name: imported\n    enabled: true\n    cmd: echo\n    args: [ready]\n' > goose.yaml
ss mcp import imported --from goose --file goose.yaml --target amp --sync -p >/dev/null
jq -e '."amp.mcpServers".imported.command == "echo"' .amp/settings.json >/dev/null
ss mcp list --json -p

Expected:

  • exit_code: 0
  • jq: .blocked == false
  • jq: .changes | length == 9

Step 8: Scripted editing and TUI opt-out

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-edit.XXXXXX")
export SKILLSHARE_CONFIG="$MCP_CASE/config.yaml"
printf 'targets: {}\n' > "$SKILLSHARE_CONFIG"
ss mcp add editable --url https://example.com/mcp --target claude --no-tui -g >/dev/null
ss mcp edit editable --url https://updated.example/mcp --no-tui -g >/dev/null
grep -q 'updated.example' "$SKILLSHARE_CONFIG"
ss mcp edit editable --url https://preview.example/mcp --dry-run --json -g >/dev/null
! grep -q 'preview.example' "$SKILLSHARE_CONFIG"
ss mcp edit editable --no-tui -g -- echo ready >/dev/null
! grep -q 'url:' "$SKILLSHARE_CONFIG"
ss mcp --no-tui -g >/dev/null
ss mcp --json -g

Expected:

  • exit_code: 0
  • jq: .blocked == false
  • jq: .changes[0].name == "editable"

Pass Criteria

All eight steps pass. Core Go tests additionally cover multi-file recovery, stale revisions, Agent-specific fields, credential references and foreign ownership. TUI model tests cover search shortcuts, hidden credentials, edit cancellation and batch import cancellation. In a real terminal, also verify mcp search/detail, edit and remove previews, multi-selection import, and the client/backup picker.

Pi extension selection and output

Inside a fresh ssenv HOME (do not install or launch either Pi extension):

set -eu
PI_CASE=$(mktemp -d "$HOME/mcp-pi.XXXXXX")
export SKILLSHARE_CONFIG="$PI_CASE/config.yaml"
printf 'targets: {}\n' > "$SKILLSHARE_CONFIG"
ss mcp add pi-docs --target pi --pi-extension pi-mcp-extension --url https://example.com/mcp --no-tui -g
ss sync mcp --dry-run --json -g | jq -e '.changes[] | select(.target == "pi" and .action == "add")'
ss sync mcp -g
jq -e '.mcpServers["pi-docs"].transport == "streamable-http"' "$HOME/.pi/agent/mcp.json"
ss sync mcp --dry-run --json -g | jq -e '.changes[] | select(.target == "pi" and .action == "unchanged")'
ss mcp edit pi-docs --pi-extension pi-mcp-adapter --no-tui -g
ss sync mcp -g
jq -e '.mcpServers["pi-docs"] | has("transport") | not' "$HOME/.pi/agent/mcp-adapter.json"
jq -e '.mcpServers | has("pi-docs") | not' "$HOME/.pi/agent/mcp.json"
ss mcp remove pi-docs --sync --no-tui -g

Expected: explicit extension choice is saved, extension output includes transport, adapter output omits it and moves from mcp.json to mcp-adapter.json, the second sync is unchanged, and removal affects only the managed entry. No MCP server or Pi package is installed or executed.

Pi built-in MCP (0.99.0+) and cleared options

set -eu
MCP_CASE=$(mktemp -d "$HOME/mcp-pi-builtin.XXXXXX")
export SKILLSHARE_CONFIG="$MCP_CASE/config.yaml"
export PI_CODING_AGENT_DIR="$MCP_CASE/pi"
printf 'targets: {}\n' > "$SKILLSHARE_CONFIG"
ss mcp add docs --target pi --pi-extension builtin --url https://example.com/mcp \
  --pi-options '{"exposure":"deferred","timeout":120,"toolExposure":{"get_*":"direct","*":"hidden"},"custom":{"flag":true}}' --no-tui -g
ss sync mcp --dry-run --json -g | jq -e '.blocked == false and (.changes[0].fields.added | length > 0)'
ss sync mcp -g
jq -e '.mcpServers.docs.exposure == "deferred" and .mcpServers.docs.transport == null' "$PI_CODING_AGENT_DIR/mcp.json"
ss mcp import --from pi --dry-run --json -g
ss mcp edit docs --pi-options '{}' --no-tui -g
ss sync mcp -g
jq -e '.mcpServers.docs.custom.flag == true' "$PI_CODING_AGENT_DIR/mcp.json"
# Re-manage the fields before testing explicit removal.
ss mcp edit docs --pi-options '{"exposure":"deferred","timeout":120}' --no-tui -g
ss sync mcp -g
ss mcp edit docs --pi-options '{}' --pi-options-prune --no-tui -g
ss sync mcp -g
jq -e '.mcpServers.docs.exposure == null and .mcpServers.docs.timeout == null and .mcpServers.docs.custom.flag == true' "$PI_CODING_AGENT_DIR/mcp.json"

Expected:

  • exit_code: 0
  • Global destination is <PI_CODING_AGENT_DIR>/mcp.json; project mode uses .pi/mcp.json.
  • Import keeps Pi-only fields. Clearing preserves values by default, explicit prune removes only owned unchanged fields.
  • No server is launched, no OAuth login or project trust is granted.