Commit Graph
30 Commits
Author SHA1 Message Date
Willie 36a314442b refactor(server): decode JSON request bodies through one size-limited helper
About 57 handlers decoded request bodies with no size limit, and the few
that had one each picked their own. They now go through decodeJSON with a
1 MiB default; handlers that need more keep their larger limit. An
oversized body gets 413 instead of whatever the handler returned for bad
JSON (MCP and plugin requests used to answer 400). Every other decode
error keeps the handler's existing status and message.
2026-09-30 02:25:42 +08:00
Willie 6a325037f2 refactor(server): share the skills config reconcile step
Install, batch install, tracked-repo install, uninstall and rehydrate
each repeated the same nine-line block to reconcile the project or
global skills config. reconcileSkillsConfig now holds that block, so a
change to reconcile behavior or logging is made once instead of five
times. Each caller still passes its own source directory, so behavior
is unchanged.
2026-09-30 01:35:10 +08:00
a352b8d243 feat(install): pin skills to a tag or commit from their web URL (#293)
* feat(install): pin GitHub tree/blob URLs to their ref

Hub entries only carry a source URL, so a team hub had no way to pin a
third-party skill to a reviewed tag or commit. The parser already read
tree/<ref>/ and blob/<ref>/ URLs but discarded the ref, installing from
the default branch instead of the version the URL names.

Keep that ref as Source.Branch. An explicit --branch (CLI) or branch
(API body) still overrides it; the server handlers now only override
when a branch is given. Batch installs group by repo and ref so two
pinned versions of one repo get separate clones.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): describe pinning hub sources to a tag or commit

Explain that a GitHub tree/<ref>/ source URL pins the install, in both
the hub index guide and the install reference.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(install): pin GitLab/Bitbucket URLs and resolve refs containing "/"

Pinning should not depend on the host, so GitLab (-/tree/<ref>/) and
Bitbucket (src/<ref>/) web URLs now keep their ref too.

Taking the first path segment as the ref broke branch names containing
"/": tree/feature/x/skills/foo cloned branch "feature" and failed. When
the first segment is not a branch or tag on the remote, the longest
matching one is used instead. A ref the remote does not have fails
with a hint to use --branch, rather than silently installing the
default branch. A saved branch that covers more of the path (from
--branch or config) moves the subdir without asking the remote.

The grouped-install round-trip test pinned -/tree/main/ against a repo
on master; the ref used to be ignored, so its branch is now renamed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): cover GitLab, Bitbucket and "/" branches in URL pinning

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(install): keep web URL refs correct on reinstall, --branch and grouping

Review of the slash-branch resolution found four gaps:

- A saved branch equal to the whole URL tail (tree/feature/x) did not
  move the subdir back, so every reinstall or update looked for "x".
- An unrelated --branch with a slash-branch URL kept the parse-time
  subdir (x/skills/foo) instead of the URL's real one.
- Grouped installs copy the source to a repo root; resolution then put
  the subdir back and discovery looked under a doubled path. Grouping
  also keyed on the unresolved first segment, so tree/feature/x and
  tree/feature/y could share one clone. Web URLs with a ref now install
  on their own, and resolution leaves a repo-root copy at its root.
- A skill named after the parse-time subdir kept that name once the
  path turned out to be all ref (tree/feature/x named the skill "x").

The unknown-ref error now reads as a missing ref first, since a stale
ref (tree/master after a rename) is more common than a slash branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(update): keep legacy installs on the default branch

Skills installed before URL refs were honoured got the default branch
and recorded no branch. Single-skill update re-parses the recorded
source, so it moved them to the URL's ref (or failed on a stale one,
with a hint to pass --branch, which update does not accept), while
update --all rebuilds from RepoURL and kept the default branch.

ApplyRecordedBranch treats an empty recorded branch as the remote
default and drops the URL's ref. It replaces the branch handling at
every site that reinstalls from recorded metadata: CLI and server
update for skills and agents, and tracked-repo rehydration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* perf(search): share one clone across pinned hub skills from a repo

Any web URL with a path after its ref installed on its own, so a hub
listing many tree/<tag>/ skills from one repo cloned it once per skill.

groupByRepo now resolves each URL's ref before grouping, with a
RemoteRefs cache that lists each remote's refs once, and groups by
repo and resolved ref. Only refs containing "/" (whose subdir the
discovery match cannot re-derive) and refs that fail to resolve still
install alone. A resolved source is marked settled so the grouped
clone does not resolve it again, and ApplyRecordedBranch settles the
subdir from a recorded branch without asking the remote.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Willie <egg358358@gmail.com>
2026-09-28 22:08:16 +08:00
Willie de1abd4b33 feat(install): pin project skills to a commit with a lockfile
config.yaml records which skills a project wants, but not which commit
each one resolved to, so two teammates running `install -p` a week apart
could end up with different skill contents.

Project mode now writes .skillshare/skills.lock.json next to the config.
`install -p` checks out the locked commit for skills and tracked repos,
and brings an already installed skill to the lock when they differ.
`update -p` and the dashboard move a pin only for skills whose commit
actually changed, so a stale local copy cannot roll the team back, and
uninstall drops the pin. A locked value is only honoured when it is a
full SHA for the same source, since it ends up as a git argument.
2026-09-21 04:31:22 +08:00
Willie efcc3e6f61 fix(update): honour installed branch in batch update and web UI check/update
Skills installed from a non-default branch record that branch in metadata,
but the grouped update path (update --all) cloned the remote default branch,
so the skills were either downgraded or reported as removed upstream and
pruned. The web UI update and check handlers had the same gap, comparing
against the remote HEAD instead of the installed branch.

Group batch updates and web UI checks by URL+branch, pass the branch through
to the clone, and resolve remote tree hashes for that branch as well.

Refs #268
2026-09-02 13:08:40 +08:00
Willie 0a3e669b09 fix(audit): decouple audit override from Force so --force reaches the gate
InstallOptions.Force carried two meanings: "overwrite what is already
there" and "proceed despite audit findings". Update paths set Force
unconditionally to overwrite a staged temp dir, so they had to leave it
off to keep the audit gate alive -- which swallowed the caller's --force
and made the gate impossible to override from either the CLI or the web
UI.

Split the audit intent into InstallOptions.AuditOverride and route every
gate through it alone. Entry points translate --force explicitly, so a
missed one degrades to "force has no effect" rather than "audit is
bypassed". Scan failures stay fail-closed regardless: accepting known
findings is not the same as being unable to learn any.

This also closes two paths that bypassed the gate outright, since both
inherited Force for overwrite reasons alone: install --json and grouped
batch updates.

In the web UI, Force Retry now appears only on failures force can
actually resolve -- a permission-denied unlink previously offered a
button that could never succeed -- and CLI-only flag hints are stripped
from messages rendered there.

Behaviour change: a --force at install time no longer exempts later
updates from the audit gate; it is a one-off decision per command.
2026-08-27 17:10:29 +08:00
Willie 54dd7d12a3 fix(update): surface missing tracked repos in update --all across CLI, API, and UI
When tracked repo metadata is committed but the gitignored clone dirs are
absent on a fresh machine, `update --all` silently reported zero work: the
batch path skipped missing repos with no message, and the web UI's
update-all paths only listed repos present on disk.

Missing tracked repos are now detected and surfaced everywhere:
- CLI: a warning block (human output) plus an aggregated
  `missing_tracked_repos` summary (names + one-shot rehydrate hint) in JSON;
  the per-item error is concise. Fixed for both global and project modes,
  and batch and single-target paths.
- Server: GET /api/update/missing-tracked-repos and the update-all/stream
  summaries report them; POST /api/update/rehydrate re-clones them from
  metadata (mirrors bare `skillshare install`).
- UI: the Update page shows a warning banner listing missing repos with a
  Rehydrate button; strings added to all 11 locales.

Closes #212
2026-06-11 15:32:28 +08:00
Willie 0e091a5e63 fix: refresh metadata after nested skill updates 2026-05-26 13:37:47 +08:00
Willie 996d00e342 feat(config): migrate consumers to Effective*Source helpers (Phase 3+5)
Make the v0.19.16 sources map actually do something: route all path
reads through Config.EffectiveSkillsSource() / EffectiveAgentsSource() /
EffectiveExtrasSource() so users who set sources.skills / sources.agents /
sources.extras get the configured paths everywhere.

Mechanical changes (~50 files):
- cmd/skillshare/**: replace cfg.Source / cfg.ExtrasSource reads with
  cfg.EffectiveSkillsSource() / cfg.EffectiveExtrasSource()
- internal/server/**: replace s.cfg.Source / s.cfg.ExtrasSource reads
  with the same helpers; server.skillsSource() / gitignoreDir() use the
  global-mode helper
- internal/config/config_validate.go: validate using EffectiveSkillsSource
  while still rejecting configs that set neither field explicitly
- internal/config/reconcile.go, registry.go: read via helpers

Behavioral fixes (paired with the migration):
- extras init / extras source / POST /api/extras no longer silently
  backfill the legacy extras_source field — the runtime fallback inside
  EffectiveExtrasSource() handles it. This stops surprise diffs in users'
  config.yaml after running extras commands.
- extras source command writes back to whichever field the user already
  configured (sources.extras if set, otherwise legacy extras_source) so
  the update is not shadowed.
- source_type in /api/extras and extras list correctly reports "default"
  vs "extras_source" by inspecting the raw (unresolved) configured value;
  sources.extras counts as configured for this purpose.

UI:
- ui/src/lib/fieldDocs.ts: neutralize sources / sources.* descriptions
  so they describe both project and global modes (Phase 5).

Preserved (intentional):
- Direct writes to cfg.Source / cfg.AgentsSource / cfg.ExtrasSource in
  init, migrate.go, cloneForSave, and server.reloadConfig — these are
  raw-field writes, not path reads.
- Updated test expectations for the removed backfills.
2026-05-21 14:49:11 +08:00
Willie 204340c2e0 fix: wire azure_hosts through project install and agent update paths
- install_project.go: pass AzureHosts to temporary Config (was only
  passing GitLabHosts, so project-mode direct install ignored azure_hosts)
- update_agents.go: thread ParseOptions through batchUpdateAgents and
  reinstallAgent, replacing bare ParseSource with ParseSourceWithOptions
- handler_update.go: use ParseSourceWithOptions with s.parseOpts() for
  agent update in web UI

Without this fix, azure_hosts in .skillshare/config.yaml was silently
ignored during project install and agent update, causing on-prem Azure
DevOps URLs to be parsed incorrectly.
2026-05-05 18:54:45 +08:00
Willie 788f9bc6e9 feat(agents): JSON output + audit gate for install/update
Extend install/update with agent-aware --json output and apply the same
audit block-threshold gate skills already use.

Install:
- install --json: imply --force; imply --all only when neither
  --skill nor --agent is set (so explicit selectors still work)
- project-mode install now routes through cmdInstallProjectParsed
  so --json suppresses UI and emits a pure-JSON payload
- agent installs propagate SourceDir so metadata lands in the
  agents root, not the install subdir
- validation error lists --agent / --kind / --branch explicitly

Update agents:
- update agents supports --force, --skip-audit, --audit-threshold
- tracked agent repos + metadata-backed single-file agents now run
  the same audit gate as skills; blocks roll back from a temp stage
- JSON output returns the final per-agent status
  (updated/skipped/failed/up_to_date/...) by merging check results
  with batch update outcomes

Audit:
- auditInstalledAgent shares one helper with auditInstalledSkill
  (scan + cleanup callbacks); single-file agent scans now use
  audit.ScanFile / ScanFileForProject instead of staging the file
  into a temp directory
- UpdateAgentFromDiscovery stages to temp, audits, then renames
  atomically (rename already overwrites on POSIX)

Refactors pulled in while reworking these paths:
- extract jsonUISuppressor helper in json_output.go and use it
  from install/update/update_agents; collapses 4 copies of the
  suppress/restore closure dance and the per-error failJSON block
- drop relPath plumbing from installAgentFromDiscoveryInternal
  now that ScanFile doesn't need a staged relative path
- list.go render functions now use theme.ANSI() so NO_COLOR is
  respected in plain-text output

Tests:
- add TestInstallAgentFromDiscovery_HighFindingBlocked (unit)
- add TestUpdate_Agents_HighFindingBlockedByThreshold (integration)
- add TestInstall_Project_JSON_Agent_PureJSON,
  TestUpdate_Agents_JSON_ReportsFinalStatus
- fix TestList_SKILLSHARE_THEME_Light / TestList_NO_COLOR to
  write a config before invoking list
- fix TestTargetList_NoTUI_FallsBackToPlainText assertion to match
  the new structured Mode: merge output
- fix TestDoctor_AllGood_PassesAll by pinning SKILLSHARE_THEME=dark
  so the no-TTY theme fallback warning doesn't fire
- add list_tui_content_test.go covering light/dark glamour base
- list detail pane now picks LightStyleConfig on light terminals

Docs:
- update install.md, update.md, understand/agents.md for the new
  flags, JSON shapes, and agents update workflow
2026-04-10 23:27:26 +08:00
Willie da000c4d07 feat(server): agent tracked repo install, prune-on-empty, audit guards
- appendAgentDiffs helper deduplicates diff/stream agent logic
- handler_agent_diff: skip local copies, junction-aware checks
- handler_sync: prune orphan agents even when source is empty
- handler_install: tracked agent repo auto-detection and routing
- handler_skills: expose RepoRelPath and Targets for agents
- handler_skills_batch: agent target assignment via frontmatter
- handler_sync_matrix: pass agent Targets to ClassifySkillForTarget
- handler_update: tracked agent repo git pull update
- handler_audit: guard empty/missing source dirs gracefully
- Tests for all of the above
2026-04-10 02:05:13 +08:00
Willie b8a189ba58 Improve resource install agent support
Add local-path resource discovery so install flows can detect agents as well as skills. Preserve nested agent paths and suffixless agent metadata keys during batch install and update, and refresh the UI copy to use resource-aware wording on the install surface.
2026-04-10 01:26:40 +08:00
Willie 5836a8802f feat: add agent filter support to Filter Studio, targets CLI, and overview API
Backend:
- PATCH /api/targets/:name accepts agent_include/agent_exclude fields
- POST /api/sync-matrix/preview returns agent entries with kind=agent
- GET /api/overview returns agentsSource and extrasSource paths
- Fix ResolveProjectTargets to resolve Agents sub-key (path + filters)
- Add agent filter CLI flags: --add-agent-include, --add-agent-exclude,
  --remove-agent-include, --remove-agent-exclude, --agent-mode
- Unify target list JSON/TUI to show agent info and filters

Frontend:
- FilterStudioPage: kind-driven single-context view (?kind=skill|agent)
- KindBadge: use design system tokens (ss-badge + wobble in playful)
- TargetsPage: link to Filter Studio with ?kind= param
- ResourceDetailPage: capitalize type labels (Local, GitHub)
- ResourcesPage: sync activeTab from URL searchParams on navigation
- DashboardPage: show agents/extras source dirs, link skills card
  with ?tab=skills for reliable tab switching
- API client: extend updateTarget and previewSyncMatrix for agent filters

Tests:
- Agent include/exclude round-trip persistence
- Invalid agent pattern returns 400
- Preview with agent entries and kind field
- No agent entries for targets without agent path
- CLI target agent filter integration tests
2026-04-09 03:50:28 +08:00
Willie 4af128fcf0 fix: migrate metadata consumers from Get(basename) to GetByPath
All call sites that looked up metadata entries using filepath.Base()
now use store.GetByPath(relPath) which handles both full-path keys
and legacy basename+group keys transparently.

Also fixes makeInstallOpts() to set SourceDir from updateContext,
preventing nested skill updates from writing metadata to wrong dir.
2026-04-08 16:35:27 +08:00
Willie d0a1df3e8b refactor: replace ReadMeta in server handler_skills and handler_update
handler_skills.go uses s.skillsStore.Get() for skill metadata.
handler_update.go uses store parameter in getServerUpdatableSkills.
2026-04-08 11:11:03 +08:00
Willie c176d89405 feat(api): add kind field to sync, audit, check, update, and install handlers
handler_sync: add Kind field to diff stream item struct.
handler_audit: add Kind field to audit stream result.
handler_check: add Kind field to check stream result.
handler_update: add Kind field to update stream result.
handler_install: add Kind field to install request for agent support.
2026-04-07 18:05:40 +08:00
Willie 657e400f8c fix(server): resolve order + registry prune for repo uninstall
- updateSingle: check exact skill path before resolveTrackedRepo
  fallback, preventing basename collision with nested repos
- handleUninstallRepo: match repo's own registry entry by FullName
  in addition to group-based matching for member skills
2026-04-04 17:05:22 +08:00
Willie a4984cd322 fix(server): resolve nested repos in update + prune registry on uninstall
P1: updateSingle now uses resolveTrackedRepo (shared with uninstall)
to find nested tracked repos like org/_team-skills instead of only
probing root-level _name/name.

P2: handleUninstallRepo now prunes registry entries belonging to the
uninstalled repo and runs ReconcileGlobalSkills/ReconcileProjectSkills,
matching the cleanup that handleUninstallBatch already does.

Extracted resolveTrackedRepo as a Server method to deduplicate the
nested path resolution logic between uninstall and update handlers.
2026-04-04 17:05:22 +08:00
Willie 0a3305f617 fix(update): capture git stderr and show actionable error messages
The update handler used internal/git which ran git commands without
capturing stderr, so failures like divergent branches only showed
"exit status 128". Now runGitWithProgress captures stderr and pipes
it through install.WrapGitError for meaningful messages.

Also adds:
- extractGitFatal() strips verbose hint: lines from git output
- Force checkbox and per-item Force retry button in Update UI
- "(try force update)" / "(try --force)" hint on pull failures
2026-03-13 09:45:56 +08:00
Willie a4d52615f7 feat(config): add gitlab_hosts for custom GitLab domain support
Self-managed GitLab instances on custom domains (e.g., git.company.com)
without "gitlab" in the hostname were parsed with 2-segment owner/repo
split instead of full-path repo. Users can now declare extra GitLab
hostnames in config so the parser treats them like gitlab.com (nested
subgroup support).

- Add `gitlab_hosts` field to both global and project Config structs
- Validate entries at load time (reject scheme, path, port, empty)
- Add `ParseOptions` / `ParseSourceWithOptions` to thread hosts into parser
- Add `isGitLabHost()` helper combining built-in + custom host detection
- Extend `InstallContext` interface with `GitLabHosts()` method
- Update all ParseSource call sites (install, search, update, server)
- Add JSON schemas for both config files
- Add unit tests for config validation and source parsing
- Add E2E runbook for devcontainer verification
- Document in configuration reference and URL formats appendix
2026-03-12 16:45:02 +08:00
Willie 38f8f1c796 fix(symlink): resolve symlinked source/target dirs across all Walk callsites
filepath.Walk/WalkDir silently skip symlinked root directories because
os.Lstat sees the root as a non-directory symlink and never enters it.
This affected update, uninstall, reconcile, and server handlers when
the source directory was a symlink (common with dotfiles managers).

Extract ResolveSymlink() from sync.go into internal/utils/path.go and
apply it at every filepath.Walk/WalkDir callsite on source/target dirs.

Also add containment guards to resolveGroupSkills (uninstall) and
resolveGroupUpdatable (update) to reject group directories that are
symlinks pointing outside the source tree.
2026-03-02 10:05:57 +08:00
Willie cd5902c590 feat: unify update audit threshold gate and docs 2026-02-23 04:18:01 +08:00
Willie bd76ec9aa5 feat: surface audit risk score in update API and UI
Return auditRiskScore and auditRiskLabel in update results so the
dashboard can show security posture after both tracked-repo and
regular-skill updates. auditGateTrackedRepo now returns the audit
result alongside the blocked indicator.
2026-02-23 04:18:01 +08:00
Willie 5ce6507799 fix: server rollback error reporting and isSecurityError classification
- handler_update.go: check git.ResetHard return value; if rollback
  fails, append explicit warning to blocked message instead of
  silently claiming rolled back
- update.go: add "post-update audit" to isSecurityError keywords
  so install.Install audit failures are correctly classified as
  security blocks in batch exit code
2026-02-23 04:18:01 +08:00
Willie a0bfe9958a fix: batch update error propagation, skip-audit passthrough, and security hardening (#39)
- P1: Batch/all update now returns non-zero when repos are blocked by
  security audit (fail-closed for CI pipelines)
- P2: Pass --skip-audit to regular skill update via InstallOptions
- P2: Add path containment check in content integrity to prevent
  directory traversal via crafted file_hashes keys
- P3: Switch git diff parsing to NUL-delimited (-z) for safe handling
  of special filenames
2026-02-23 04:18:01 +08:00
Willie 3b8c681bb2 feat: auto-inject auth tokens for hub search and Web UI git operations
Add token auto-injection for private repo support across all platforms:
- Hub search (--hub): normalize web URLs to raw-content URLs and apply
  platform-specific auth headers (GitLab, GitHub, Bitbucket)
- Web UI push/pull/update: use *WithAuth variants for git operations
- Export ResolveTokenForURL and DetectPlatformForURL from install/auth
2026-02-22 15:13:10 +08:00
Willie 3a132ab72e fix: update nested skills by full relative path in Web UI
The Update button on SkillDetailPage passed only the skill basename
(e.g. "ascii-box-check") instead of the full relative path
("utils/ascii-box-check"), causing "not found as tracked repo or
updatable skill" errors for nested skills.

- Frontend: use skill.relPath instead of skill.name for non-repo skills
- Backend: rewrite getServerUpdatableSkills with filepath.WalkDir so
  Update All also discovers nested skills
2026-02-13 14:21:36 +08:00
Willie e5813cc7e0 feat: add oplog instrumentation to all API handlers, improve audit/log/sync CLI
- Add oplog write helpers to server (oplog_helper.go) and instrument
  all handler_*.go endpoints with ops/audit logging
- Improve audit CLI with structured auditRunSummary and severity counts
- Enhance log CLI with richer filtering and detail display
- Improve sync CLI with drift detection summaries in oplog
- Reorder web UI sidebar nav by user workflow (core → acquire → distribute → maintain → monitor)
- Enhance LogPage with better formatting and filtering
- Add server oplog unit tests (logging_test.go, log_test.go)
- Update docs for log command and quick-reference
2026-02-10 03:50:03 +08:00
Willie 092fb14f8a feat: add web dashboard with embedded React SPA (skillshare ui)
Single-binary web UI for visual skill management — React 19 + Vite
frontend embedded via go:embed, Go net/http REST API backend (30+
endpoints). Includes dashboard, skills browser, targets, sync, collect,
search, install, config editor, backup/restore, and git sync pages.

Also adds Bitbucket/GitLab URL parsing for install, git operations
library, skill version checking, CI frontend build steps, Docker
sandbox support, and website docs for the ui command.
2026-02-08 05:08:30 +08:00