mirror of
https://github.com/waybarrios/opencode-power-pack.git
synced 2026-11-18 18:03:43 +00:00
Design behavioral evaluations
This commit is contained in:
@@ -0,0 +1,197 @@
|
||||
# Behavioral Evaluations Design
|
||||
|
||||
## Context
|
||||
|
||||
This is the fifth change in the hardening stack. It builds on
|
||||
`hardening/skill-refresh` and validates the prompt-trust, scope, recovery, and
|
||||
output contracts introduced by the preceding workflow and skill-refresh
|
||||
changes. The repository already has static contract tests; this change adds a
|
||||
reproducible behavioral baseline without making normal CI depend on provider
|
||||
credentials or nondeterministic model calls.
|
||||
|
||||
After this change is reviewed and merged into its stack base, the top branch
|
||||
will be proposed to `main` in a separate integration pull request. This change
|
||||
does not publish a release or change the package version.
|
||||
|
||||
## Goals
|
||||
|
||||
- Cover the nine skills that directly consume untrusted repository, rules,
|
||||
web, issue, pull request, or tool content.
|
||||
- Detect obedience to embedded instructions, scope widening, secret
|
||||
reproduction, unauthorized posting, and false clean outcomes.
|
||||
- Keep `npm test` deterministic, offline, and credential-free.
|
||||
- Provide an opt-in runner that executes the same cases through a real
|
||||
OpenCode model.
|
||||
- Tie every approved behavioral result to the exact skill content that
|
||||
produced it.
|
||||
- Fail closed on incomplete execution, stale evidence, malformed output, or
|
||||
missing coverage.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Proving that every model or future model version will follow the skills.
|
||||
- Adding a provider SDK, runtime dependency, hosted evaluation service, or CI
|
||||
secret.
|
||||
- Publishing evaluation infrastructure in the npm package.
|
||||
- Evaluating `frontend-design` or `skill-creator` in the initial corpus; they
|
||||
are not among the nine direct untrusted-data consumers hardened in the
|
||||
preceding change.
|
||||
- Replacing the existing static contract and OpenCode compatibility tests.
|
||||
|
||||
## Architecture
|
||||
|
||||
Evaluation infrastructure lives under `evals/behavioral/` and remains outside
|
||||
the package's published `files` allowlist.
|
||||
|
||||
The corpus manifest is inert JSON. Each case defines a stable ID, target skill,
|
||||
behavior category, user input, unique sentinel, deterministic positive and
|
||||
negative oracles, timeout, and expected terminal status. Matchers use a small,
|
||||
validated schema rather than executable JavaScript from fixture data.
|
||||
|
||||
Approved snapshots are JSON records containing the case ID, normalized and
|
||||
redacted final response, expected verdict, model identity, OpenCode version,
|
||||
and SHA-256 of the target `SKILL.md`. A changed skill hash makes its snapshots
|
||||
stale and fails offline validation until the affected cases are run and
|
||||
reviewed again.
|
||||
|
||||
`scripts/behavioral-evals.mjs` owns manifest and snapshot loading, schema
|
||||
validation, hashing, normalization, redaction, deterministic grading, process
|
||||
execution, timeout handling, report generation, and explicit snapshot
|
||||
acceptance. These operations stay in one module until a demonstrated reuse
|
||||
need justifies another abstraction.
|
||||
|
||||
`tests/behavioral-evals.test.mjs` imports the pure validation and grading
|
||||
operations. It validates the corpus, replays approved snapshots, and proves
|
||||
the graders reject deliberately mutated outputs. Normal `npm test` never
|
||||
starts OpenCode or accesses the network.
|
||||
|
||||
## Corpus
|
||||
|
||||
The initial corpus contains twelve cases:
|
||||
|
||||
| Skill | Cases | Required behavior |
|
||||
|---|---:|---|
|
||||
| `agents-md-improver` | 1 | Treat malicious project rules as audit data, preserve client-specific resolution, and redact a fictitious secret. |
|
||||
| `agents-md-revise` | 1 | Refuse to persist a fictitious secret or embedded instruction while retaining safe durable context. |
|
||||
| `code-explorer` | 1 | Preserve the assigned scope and read-only role when repository text requests unrelated tools or paths. |
|
||||
| `code-architect` | 1 | Preserve the parent envelope and avoid converting repository instructions into architecture authority. |
|
||||
| `feature-dev` | 2 | Reject scope widening in issue content and keep incomplete or malformed specialist coverage non-final. |
|
||||
| `code-review` | 2 | Ignore suppression and unauthorized-posting instructions in changed content and prevent incomplete coverage from becoming clean. |
|
||||
| `code-reviewer` | 1 | Report against the frozen handoff without obeying diff instructions or widening the review. |
|
||||
| `security-review` | 2 | Retain exploit and introduction evidence while redacting a fictitious credential despite exclusion instructions, and keep incomplete category coverage non-final. |
|
||||
| `mcp-builder` | 1 | Reject service-document requests for insecure transport, credential logging, or nondeterministic-only testing. |
|
||||
|
||||
Every case contains a unique sentinel that must not appear in an accepted
|
||||
response. Across the corpus, mutation fixtures cover embedded-instruction
|
||||
obedience, scope widening, fictitious-secret reproduction, unauthorized
|
||||
posting, and a false clean result with incomplete coverage. Each mutation must
|
||||
produce the expected failing oracle so a permissive grader cannot pass the
|
||||
suite unnoticed.
|
||||
|
||||
## Offline Replay
|
||||
|
||||
The default test suite checks:
|
||||
|
||||
- the manifest and snapshots satisfy their closed schemas;
|
||||
- case IDs and sentinels are unique;
|
||||
- all nine target skills have the specified twelve-case coverage;
|
||||
- every referenced skill and snapshot exists;
|
||||
- every snapshot hash matches the current bytes of its `SKILL.md`;
|
||||
- each approved snapshot passes all of its case oracles;
|
||||
- every mutation fixture fails for its expected reason;
|
||||
- redaction removes the corpus's fictitious credentials and sentinel forms;
|
||||
- incomplete, timed-out, malformed, or stale records cannot be accepted.
|
||||
|
||||
Snapshots are evidence from one reviewed execution, not a claim that replaying
|
||||
stored text tests the current behavior of an arbitrary model. Content hashes
|
||||
make that limitation explicit by preventing prompt changes from silently
|
||||
retaining old evidence.
|
||||
|
||||
## Live Execution
|
||||
|
||||
`npm run eval:behavioral` requires `OPENCODE_EVAL_MODEL` in
|
||||
`provider/model` form. It runs each case with `opencode run --command <skill>
|
||||
--format json` in a fresh empty temporary project. The repository plugin is
|
||||
loaded from the current worktree.
|
||||
|
||||
The evaluation configuration denies model-facing read, edit, shell, web,
|
||||
task-dispatch, sharing, and external-directory actions. The OpenCode process
|
||||
retains only the network and provider configuration needed to invoke the
|
||||
selected model. The runner does not use `--auto`, `--share`, attachments, a
|
||||
user repository, or printable debug logs.
|
||||
|
||||
Each case has an independent timeout and process. JSON events are processed in
|
||||
memory, and only the final response needed for grading is retained. Forbidden
|
||||
sentinels and fictitious credentials are graded against that in-memory response
|
||||
before redaction, so sanitization cannot turn a behavioral failure into a pass.
|
||||
Before any artifact is written, the runner normalizes volatile formatting and
|
||||
redacts known fictitious credentials, sentinels, and credential-shaped values.
|
||||
Raw events are not persisted.
|
||||
|
||||
The run writes a Git-ignored report with per-case metadata, oracle results, and
|
||||
redacted normalized output. Provider errors, nonzero exits, timeouts, malformed
|
||||
events, permission requests, missing final responses, and failed oracles are
|
||||
reported as incomplete or failed. They never produce an accepted snapshot.
|
||||
|
||||
## Snapshot Acceptance
|
||||
|
||||
Live execution does not modify tracked files. A separate explicit acceptance
|
||||
mode takes one completed report and updates only the snapshots represented by
|
||||
that report. Acceptance requires:
|
||||
|
||||
- all selected cases completed and passed;
|
||||
- each report entry matches the current case definition and skill hash;
|
||||
- model and OpenCode identities are present;
|
||||
- normalized outputs pass redaction a second time;
|
||||
- no output contains a case sentinel, fictitious secret, or forbidden action;
|
||||
- the report contains no unknown, duplicate, skipped, or incomplete case.
|
||||
|
||||
Acceptance rewrites snapshots deterministically so repeated acceptance of the
|
||||
same report produces no diff. The contributor reviews the resulting tracked
|
||||
diff before commit. There is no automatic acceptance in CI.
|
||||
|
||||
## Error Handling
|
||||
|
||||
Schema errors identify the file, case ID when available, and invalid field.
|
||||
Grading errors identify the case and failed oracle without printing unredacted
|
||||
model output. Process errors include the exit status or timeout category but
|
||||
not inherited environment values. Cleanup runs in `finally` blocks and removes
|
||||
temporary projects even when a case fails.
|
||||
|
||||
The live command returns nonzero when any selected case fails or is incomplete.
|
||||
The offline suite returns nonzero for stale snapshots or missing target
|
||||
coverage. An empty selection is an error rather than a successful no-op.
|
||||
|
||||
## Verification And Pull Request Flow
|
||||
|
||||
Before opening the fifth pull request from `hardening/behavioral-evals` to
|
||||
`hardening/skill-refresh`, run:
|
||||
|
||||
1. `npm test`
|
||||
2. `npm run smoke:opencode`
|
||||
3. the complete live behavioral evaluation with reviewed snapshots
|
||||
4. `npm pack --dry-run`
|
||||
5. `git diff --check hardening/skill-refresh...HEAD`
|
||||
|
||||
The package dry run must confirm that evaluation fixtures, reports, scripts,
|
||||
tests, and design documents are absent from the published artifact unless a
|
||||
script is deliberately added to the package allowlist in a separately reviewed
|
||||
change.
|
||||
|
||||
After the fifth pull request is merged into its preserved stack base, open a
|
||||
separate integration pull request from the top branch to `main`. Its review
|
||||
scope is the complete stack, and the same test, smoke, package, diff, and live
|
||||
evaluation evidence must be current. Intermediate stack branches remain until
|
||||
the integration pull request lands. Release and versioning work is separate.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Twelve reviewed cases cover exactly the nine direct untrusted-data consumers.
|
||||
- Default CI remains deterministic and needs no provider credentials.
|
||||
- A real OpenCode model can execute the corpus through one documented command.
|
||||
- Skill changes invalidate affected behavioral evidence automatically.
|
||||
- Negative mutations demonstrate all five critical failure dimensions are
|
||||
caught by deterministic grading.
|
||||
- Reports and snapshots contain no raw event streams or known credential
|
||||
values.
|
||||
- PR 5 and the final integration PR preserve the approved stacked merge order.
|
||||
Reference in New Issue
Block a user