Commit Graph
23 Commits
Author SHA1 Message Date
Willie 9c2dcda9e6 feat(runbook): retry/depends directives, remove TUI, add skip reasons
Add #112 step directive framework (retry, delay, depends) via HTML
comment syntax: <!-- runbook: retry=3 delay=5s depends=2 -->

- Parser: applyDirectiveComment() parses retry/delay/depends/timeout
- Executor: retry wraps subshell in for-loop, depends checks
  __rb_status_N, exit code -2 sentinel for depends-skipped
- Fix __rb_rc stale value bug when depends-else + fail-fast interact
- Reporter: show skip reasons (depends/fail-fast) at all verbosity levels

Remove TUI (executor.go, reporter_tui.go) to prepare for independent
extraction. All execution now uses unified session executor path,
eliminating feature parity issues (retry/depends/env were TUI-only gaps).

Tests: +10 new tests covering directive parsing, retry, depends,
fail-fast, env seeding, per-step timeout at both executor and runner
levels.
2026-03-11 04:53:08 +08:00
Willie 65fc6a0458 feat(runbook): per-step timeout directive and env seeding from config
- Per-step timeout: `### Step 5: Build (timeout: 10m)` overrides the
  global timeout for individual steps. Uses `timeout` command + heredoc
  in the session script to avoid quoting issues.
- Env seeding: `runbook.json` `env` field injects environment variables
  into all steps via `export` at the top of the session script.
  Deterministic key ordering via sortedKeys().
2026-03-11 04:53:08 +08:00
Willie a80eaddd1a feat(runbook): --fail-fast, verbose (-v/-vv), and --output flags
Three DX improvements for runbook runner:

- --fail-fast: two-layer stop-on-failure (bash __rb_stop flag +
  Go assertion loop) skips remaining steps after first failure,
  reducing noise in long runbooks
- -v/-vv: verbose output showing assertion details (level 1) and
  stdout/stderr snippets (level 2) for easier debugging
- --output/-o: write JSON report to file while still printing
  plain text summary to stdout, useful for CI artifact collection
2026-03-11 04:53:08 +08:00
Willie b932ddae78 feat(runbook): selective step execution with --steps and --from flags
Skip re-running unchanged steps when debugging runbooks.
--steps 1,3,5 runs only specified steps; --from N runs from step N onwards.
Filtered steps are marked skipped. Flags are mutually exclusive.
2026-03-11 04:53:08 +08:00
Willie a9053eea11 feat(runbook): regex (?m) default, heredoc-aware parser, remove set -e
Three runbook CLI improvements addressing top pain points from
verification:

- Auto-prepend (?m) to regex assertions so ^/$ match line boundaries
  instead of string boundaries (skip if pattern already has flags)
- Track heredoc delimiters in parser so embedded ``` inside heredocs
  don't prematurely close code blocks
- Remove set -e from step subshells — exit code is now the last
  command's exit code, eliminating need for CMD_EXIT workarounds

Resolves opportunities #103, #105, #106.
2026-03-11 04:53:08 +08:00
Willie ebab63a670 feat(runbook): add build lifecycle hook, verify all 17 runbooks
Add a build hook that runs once before all runbooks, enabling
automatic binary rebuild. Lifecycle is now:
  build (once) → [per runbook: setup → steps → teardown]

Build hook uses os/exec directly (not session executor), outputs
to stderr, and aborts immediately on failure.

Fix all 17 runbooks to pass in devcontainer:
- Adapt to session executor's set -eo pipefail (use cmd || EXIT=$?
  pattern for expected non-zero exits)
- Replace embedded ```bash in heredocs with printf to avoid
  confusing the runbook parser
- Fix negated assertion gotchas (Not FAIL, No such file)
- Remove fragile verify code blocks (multi-block merge issues)
- Fix step number collision (7b → 8, renumber subsequent)
- Use -s flag for batch installs instead of repeated --force
- Add (?m) flag for multi-line regex assertions
2026-03-11 04:53:08 +08:00
Willie 06e9783eb4 fix(runbook): hooks work in TUI mode, show status in plain reporter
Disable TUI when hooks are active (hooks need session executor for
env persistence). Show [setup]/[teardown] status in plain text output.
Fix negative step numbers in marker regex for synthetic steps.
2026-03-11 04:53:08 +08:00
Willie 2657974fbe feat(runbook): lifecycle hooks via --setup/--teardown and runbook.json
Add setup and teardown commands that run before/after each runbook.
Config file (runbook.json) provides persistent defaults; CLI flags
override. Setup env vars persist to all steps via session executor.
Setup failure skips all steps; teardown failure is informational only.

Removed Step 0 from atomgit runbook — init is now in runbook.json.
2026-03-11 04:53:08 +08:00
Willie f97c67c4b6 feat(runbook): typed assertion engine with exit_code, regex, jq support
Prefix-based dispatch: exit_code: N, regex: <pattern>, jq: <expr>
fall through to substring match. Assertions always run when defined
and determine pass/fail regardless of exit code. Supports negation
(!0 for exit_code, Not/NOT for substring).
2026-03-11 04:53:08 +08:00
Willie 1c23571aa2 feat(runbook): session executor preserves shell variables across steps
Single bash process runs all auto steps sequentially. Each step uses a
subshell with EXIT trap to persist env via export -p. Marker-based
output parsing splits per-step stdout/stderr. Existing runbooks work
without rewriting.
2026-03-11 04:53:08 +08:00
Willie a988309e89 feat(runbook): enforce container-only execution with safety checks
Three-layer defense: main.go gate, ExecuteSession check, and Execute
check all verify /.dockerenv or RUNBOOK_ALLOW_EXECUTE before running
commands. Prevents accidental host-machine execution.
2026-03-11 04:53:08 +08:00
Willie 910a8d548f refactor(runbook): consolidate duplicate helpers and use status constants
Extract shared helpers (truncateText, formatDurationMs, stepFailReason,
checkAssertions) and status/executor constants into helpers.go. Remove
7 duplicate functions across reporter_tui.go and reporter_plain.go.
Replace all magic strings with typed constants for compile-time safety.
2026-03-11 04:53:08 +08:00
Willie 7d4a6fd282 fix(runbook): address code review findings
- Guard TUI Init() against empty steps (prevents panic)
- Fix plainFailReason for negated assertions (align with TUI logic)
- Add pipefail to bash executor for pipeline error detection
- Remove unused RunOptions.TUI field
2026-03-11 04:53:08 +08:00
Willie 336bdb9e25 test(runbook): integration tests with self-contained and real runbook parsing
Three integration tests:
- SelfContainedRunbook: 3 bash steps through RunRunbook, verifies 3/3 pass + valid JSON
- ParseAllRealRunbooks: parses all 17 runbooks, checks titles/steps/classification
- DryRunAllRunbooks: dry-runs all 17 runbooks, verifies all steps skipped
2026-03-11 04:53:08 +08:00
Willie a1ba7eae6e feat(runbook): plain text reporter for single and batch output
WriteSingleReport prints per-step results with status icons,
duration, and failure reasons. WritePlainSummary aggregates
multiple runbook results into a compact batch summary.
2026-03-11 04:53:08 +08:00
Willie e53ca07639 feat(runbook): CLI entry with --report json, --dry-run, --no-tui flags
Replace placeholder main.go with full CLI that supports:
- Single file or directory of *_runbook.md files
- TUI mode (default for single files), plain text, and JSON output
- --dry-run for parse-and-classify without execution
- Per-step timeout with --timeout flag
- Exit code 1 on any failure
2026-03-11 04:53:08 +08:00
Willie a4343df41d feat(runbook): runner engine wires parser, classifier, executor, and assertions 2026-03-11 04:53:08 +08:00
Willie 67ceed3270 feat(runbook): JSON report writer 2026-03-11 04:53:08 +08:00
Willie 377aa0bafc feat(runbook): assertion matcher with negation and case-insensitive matching 2026-03-11 04:53:08 +08:00
Willie f919bfb7ae feat(runbook): bash executor with stdout/stderr capture and timeout
Execute() runs step commands via bash -e -c, captures stdout/stderr
separately, extracts exit codes via syscall.WaitStatus, and handles
context timeout/cancellation. Merged code blocks (---) are joined
before execution. Six tests cover echo, exit codes, multiline,
timeout, stderr capture, and merged blocks.
2026-03-11 04:53:08 +08:00
Willie c848309c78 feat(runbook): step classifier categorizes auto/manual execution 2026-03-11 04:53:08 +08:00
Willie 173c09d67b feat(runbook): markdown parser extracts steps, code blocks, and expected assertions
Line-by-line state machine parser (bufio.Scanner + regex) that handles all
17 real E2E runbook files. Supports multiple heading formats (## Step N,
### N., ### Nb., double-labeling), multi code blocks per step (merged with
\n---\n separator), Expected/bold-Expected variants, inline markdown
stripping, and stops at Pass/Fail Criteria sections.
2026-03-11 04:53:08 +08:00
Willie b2a9ebbaab feat(tools): scaffold runbook test runner with types 2026-03-11 04:53:08 +08:00