Commit Graph
24 Commits
Author SHA1 Message Date
Willie 37fec01d34 fix(search): accept every skill name that install accepts
Search kept its own name check, which rejected mixed-case names such as
"MySkill" that install happily accepts, so those skills never showed up
in results. It now uses validate.SkillName, the rule install applies.
Names install rejects, such as a leading underscore, stay hidden.
2026-09-30 02:19:47 +08:00
a352b8d243 feat(install): pin skills to a tag or commit from their web URL (#293)
* feat(install): pin GitHub tree/blob URLs to their ref

Hub entries only carry a source URL, so a team hub had no way to pin a
third-party skill to a reviewed tag or commit. The parser already read
tree/<ref>/ and blob/<ref>/ URLs but discarded the ref, installing from
the default branch instead of the version the URL names.

Keep that ref as Source.Branch. An explicit --branch (CLI) or branch
(API body) still overrides it; the server handlers now only override
when a branch is given. Batch installs group by repo and ref so two
pinned versions of one repo get separate clones.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): describe pinning hub sources to a tag or commit

Explain that a GitHub tree/<ref>/ source URL pins the install, in both
the hub index guide and the install reference.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(install): pin GitLab/Bitbucket URLs and resolve refs containing "/"

Pinning should not depend on the host, so GitLab (-/tree/<ref>/) and
Bitbucket (src/<ref>/) web URLs now keep their ref too.

Taking the first path segment as the ref broke branch names containing
"/": tree/feature/x/skills/foo cloned branch "feature" and failed. When
the first segment is not a branch or tag on the remote, the longest
matching one is used instead. A ref the remote does not have fails
with a hint to use --branch, rather than silently installing the
default branch. A saved branch that covers more of the path (from
--branch or config) moves the subdir without asking the remote.

The grouped-install round-trip test pinned -/tree/main/ against a repo
on master; the ref used to be ignored, so its branch is now renamed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): cover GitLab, Bitbucket and "/" branches in URL pinning

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(install): keep web URL refs correct on reinstall, --branch and grouping

Review of the slash-branch resolution found four gaps:

- A saved branch equal to the whole URL tail (tree/feature/x) did not
  move the subdir back, so every reinstall or update looked for "x".
- An unrelated --branch with a slash-branch URL kept the parse-time
  subdir (x/skills/foo) instead of the URL's real one.
- Grouped installs copy the source to a repo root; resolution then put
  the subdir back and discovery looked under a doubled path. Grouping
  also keyed on the unresolved first segment, so tree/feature/x and
  tree/feature/y could share one clone. Web URLs with a ref now install
  on their own, and resolution leaves a repo-root copy at its root.
- A skill named after the parse-time subdir kept that name once the
  path turned out to be all ref (tree/feature/x named the skill "x").

The unknown-ref error now reads as a missing ref first, since a stale
ref (tree/master after a rename) is more common than a slash branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(update): keep legacy installs on the default branch

Skills installed before URL refs were honoured got the default branch
and recorded no branch. Single-skill update re-parses the recorded
source, so it moved them to the URL's ref (or failed on a stale one,
with a hint to pass --branch, which update does not accept), while
update --all rebuilds from RepoURL and kept the default branch.

ApplyRecordedBranch treats an empty recorded branch as the remote
default and drops the URL's ref. It replaces the branch handling at
every site that reinstalls from recorded metadata: CLI and server
update for skills and agents, and tracked-repo rehydration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* perf(search): share one clone across pinned hub skills from a repo

Any web URL with a path after its ref installed on its own, so a hub
listing many tree/<tag>/ skills from one repo cloned it once per skill.

groupByRepo now resolves each URL's ref before grouping, with a
RemoteRefs cache that lists each remote's refs once, and groups by
repo and resolved ref. Only refs containing "/" (whose subdir the
discovery match cannot re-derive) and refs that fail to resolve still
install alone. A resolved source is marked settled so the grouped
clone does not resolve it again, and ApplyRecordedBranch settles the
subdir from a recorded branch without asking the remote.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Willie <egg358358@gmail.com>
2026-09-28 22:08:16 +08:00
Willie e4ba03334a chore: release v0.20.8 2026-06-05 13:56:07 +08:00
Willie 2b45618f74 feat(search): clearer hub load errors with structured toast
Hub fetch failures surfaced only a bare "fetch hub: HTTP 400" toast, with no indication of which hub failed or how to fix it.

Hub load errors now name the likely cause and include the URL for every failure mode (4xx/auth, malformed URL, invalid JSON, unreachable host, missing local file). The Search page renders them as a structured toast titled with the failing hub, and Toast gained an optional title for this.
2026-06-03 15:38:17 +08:00
Willie 694d784799 fix(preview): render SKILL.md for non-github.com hub sources
The web UI skill preview always fetched SKILL.md via api.github.com, so skills from GHE or other non-github.com hubs showed only index metadata (name, description, tags) instead of the full rendered content.

FetchPreview now resolves the source platform: github.com and GHE use the host's REST Contents API, while other platforms (GitLab, Bitbucket, Azure, SSH) fall back to a shallow clone. Sources whose content can't be retrieved degrade gracefully with a clear notice in the modal.

Fixes #193
2026-06-03 15:37:57 +08:00
Willie 1d0ee69955 feat: support SSH hub sources for private and GHE hosts
Hub index fetch previously handled only unauthenticated HTTP(S) GET or
local file paths, leaving private and GitHub Enterprise repos out of
reach — raw HTTPS URLs redirect to login before token headers apply, and
SSH-style hub URLs were unrecognized.

Resolve SSH hub sources by shallow-cloning the repo with the user's SSH
keys and reading the index from it. The in-repo index path uses the
existing //path subdir convention (consistent with `install`), defaulting
to skillshare-hub.json at the repo root. Both scp-style and ssh:// URLs
are accepted. In the web dashboard, SSH clones are restricted to saved
hubs so a request-supplied URL can't drive an arbitrary clone.

Ref #185
2026-06-02 16:29:25 +08:00
Willie 0c2981d24d feat(search): add repo-scoped results and inline frontmatter parsing
- Add repositoryMatchScore so popular repos matching the query (e.g. 'superpowers')
  rank above lower-signal exact-name copies; rebalance weights to fit.
- Fetch additional skill candidates via GitHub repository search + git tree API,
  surfacing SKILL.md files from well-known repos that code search misses.
- Prioritize preferred skill repos when batching star enrichment so their star
  counts are always populated within the fetch cap.
- Parse one-line '--- name: ... description: ... ---' frontmatter so single-line
  SKILL.md files are discoverable and render correct metadata.
- Update CHANGELOG/website changelog for v0.19.5.
2026-04-23 18:11:37 +08:00
Willie 07c091529b refactor(search): harden preview error handling, cache, and a11y
- Add ErrSkillNotFound sentinel error in search/preview.go so callers
  can distinguish 404 from other errors with errors.Is instead of
  brittle string matching
- Cap previewCache at 200 entries with TTL purge + full reset on
  overflow to prevent unbounded memory growth
- Make Card and SearchResultsTable rows keyboard-accessible
  (role=button, tabIndex=0, Enter/Space handlers)
- Refactor SkillPreviewModal to accept a single SearchResult prop
  instead of 6 individual fields — reduces prop drilling and keeps
  the modal in sync with the result shape
- Fix search_online_test: add --list flag so output is not gated
  behind interactive TUI
2026-04-23 16:46:53 +08:00
Willie 4c5c56bb05 feat(search): improve result quality with rebalanced scoring and quality filters
Search results were dominated by low-quality repos because the scoring
formula gave name match 45% weight while stars (the strongest quality
proxy) only had 5%. Zero-star personal experiments ranked above
well-known skill repos.

Changes:
- Rebalance scoring weights: name 30%, desc 20%, stars 25%, source 25%
- Improve normalizeStars curve (log10/3.0) for better differentiation
  in the 5-50 star range where most useful skills live
- Lower sourceQualityScore base from 0.35 to 0.15 for unknown repos
- Add filterLowQuality(): remove 0-star non-preferred repos, short
  descriptions (<10 chars), and known spam orgs (inference-sh)
- Refactor search pipeline: extract buildGitHubCodeSearchQuery(),
  processSearchItems(), convertToSearchResult() for testability
- Add preferred repo boosting (anthropics/skills, vercel-labs/skills)
- Add SKILL.md frontmatter validation to filter non-skill results
- Add equivalent skill deduplication (same name+description → keep
  best source)
- Add repo-scoped query support (owner/repo[/subdir] patterns)

Closes #130 in .feature-radar/opportunities/
2026-04-23 16:26:14 +08:00
Willie a3235473eb feat: add skill preview modal to Search page (#141)
Add a preview modal so users can read a skill's SKILL.md content
before installing. Clicking a search result card or table row opens
the modal; the Install button is now inside the modal.

Backend:
- New GET /api/preview?source=owner/repo/path endpoint
- FetchSkillContent fetches SKILL.md from GitHub Contents API
- Git Tree API fallback for hub sources where the install shorthand
  doesn't match the repo's actual file path
- In-memory TTL cache (5 min) via sync.Map
- Proper 404/429/500 error handling with rate-limit support

Frontend:
- SkillPreviewModal component (DialogShell-based, lazy-fetch)
- Cards and table rows clickable to open preview
- Graceful fallback: if fetch fails but hub metadata exists, show
  description + tags instead of error
- Hide Stars column in table view for hub mode
- Fix Card component missing onClick prop passthrough
- Fix IntersectionObserver not reconnecting after loading more items
- Prevent background scroll when any modal is open (DialogShell)
- i18n keys added to all 11 locales

Closes #141
2026-04-23 14:20:42 +08:00
Willie d9ee14aa42 feat(search): add FetchSkillContent for remote skill preview
Fetches full SKILL.md from GitHub Contents API, parses frontmatter
metadata (name, description, license, tags), and returns star count.
Reuses existing ghclient auth and rate-limit patterns.

Ref: #141
2026-04-23 09:57:22 +08:00
Willie ae16181689 refactor(auth): consolidate duplicated auth infra and fix GH_TOKEN gap
- Remove duplicated RateLimitError, newGitHubRequest, newGitHubClient,
  checkRateLimit from internal/version — now uses internal/github client,
  which also fixes missing `gh auth token` fallback for version checks
- Add GH_TOKEN as fallback in install.resolveToken() for GitHub platform,
  matching github.GetToken() resolution order (GITHUB_TOKEN → GH_TOKEN)
- Add GH_TOKEN to sanitizeTokens() redaction list
- Extract shared IsAuthError() in install/auth.go; push.go and
  wrapGitError() both delegate to it (adds "Access denied" coverage)
- Consolidate shouldFallback*TrackedClone auth checks via isAuthOrAccessError()
- Thread single *http.Client through search.Search() and all callees,
  reducing ~60 redundant TLS handshakes to connection-pooled reuse
- Pass *http.Client to downloadFile() instead of creating per-file clients
- Replace sourceHost() with extractHost() (same package, identical purpose)
- Remove unused isGitRepo() alias; callers use IsGitRepo() directly
2026-03-02 02:23:50 +08:00
Willie b014f8c6dc feat: optimize large-repo install and update flows
Implement Issue #46 across install/update paths to reduce perceived hangs and improve subdir installation performance on large repositories.

Main changes:
- Add git progress streaming primitives in internal/install/git_progress.go.
  - runGitCommandWithProgress supports stderr streaming callbacks in TTY mode.
  - scanGitProgress handles both CR and LF progress delimiters.
- Thread InstallOptions.OnProgress through clone/pull/discovery/install/update paths.
  - Keep non-TTY behavior quiet to avoid noisy logs.
  - Add TreeSpinner.Update for live message updates.
- Add progress-aware git update APIs in internal/git/info.go:
  - PullWithProgress and ForcePullWithProgress (keep PullWithEnv/ForcePullWithEnv for compatibility).
- Introduce shared GitHub client utilities in internal/github/client.go.
  - Centralize token lookup, request construction, and rate-limit handling.
  - Refactor internal/search/search.go to use shared client while preserving compatibility aliases.
- Add GitHub/GHE API subdir fast path in internal/install/github_download.go.
  - github.com => api.github.com
  - github-like enterprise hosts => https://<host>/api/v3
  - On API failure, emit warning and fallback to git-based flows.
- Add sparse checkout fallback for non-GitHub subdir sources in internal/install/sparse_checkout.go.
  - Use filter+no-checkout+depth=1 then sparse-checkout set + checkout.
- Extend discovery/install metadata plumbing.
  - DiscoveryResult now includes CommitHash and Warnings.
  - install/discover subdir paths preserve warnings and commit hashes from API/sparse/full-clone flows.
- Preserve compatibility and UX expectations.
  - Keep existing action strings (for tests/UI stability).
  - Keep legacy discover/pull APIs as wrappers around new progress-aware variants.
- Improve --track download efficiency.
  - Track clone now attempts --filter=blob:none --depth 1 --single-branch first.
  - Falls back to full clone only on remote capability mismatch, not on auth/not-found errors.

CLI wiring:
- cmd/skillshare/install.go
  - TTY handlers now pass progress callbacks and use with-progress discover APIs.
  - Subdir handler shows API-vs-clone stage messages and renders discovery fallback warnings.
- cmd/skillshare/update.go and cmd/skillshare/update_project.go
  - Use PullWithProgress/ForcePullWithProgress for tracked repos.
  - Ensure reinstall/update install flows forward progress callbacks.

Tests added/updated:
- internal/install/git_progress_test.go
- internal/install/github_download_test.go
- internal/install/sparse_checkout_test.go
- internal/install/tracked_clone_test.go
- internal/install/source_test.go (GitHubOwner/GitHubRepo coverage)
- internal/git/info_test.go (PullWithProgress/ForcePullWithProgress)
- tests/integration/install_subdir_optimization_test.go
- tests/integration/install_online_test.go (GitHub API subdir case)

Validation performed:
- go test ./internal/install -count=1
- go test ./internal/git -run 'Test.*Pull.*' -count=1
- go build -o bin/skillshare ./cmd/skillshare && SKILLSHARE_TEST_BINARY="/Users/williehung/Developer/Apps/github/skillshare/bin/skillshare" go test ./tests/integration -run 'TestInstall_Subdir|TestUpdate_' -count=1
- go build -o bin/skillshare ./cmd/skillshare && SKILLSHARE_TEST_BINARY="/Users/williehung/Developer/Apps/github/skillshare/bin/skillshare" go test ./tests/integration -run 'TestInstall_Track_|TestUpdate_TrackedRepo' -count=1
- go test -tags online ./tests/integration -run TestInstall_GitHubSubdirViaAPI -count=1
- make lint
- make test
2026-02-25 07:02:54 +08:00
Willie 3b8c681bb2 feat: auto-inject auth tokens for hub search and Web UI git operations
Add token auto-injection for private repo support across all platforms:
- Hub search (--hub): normalize web URLs to raw-content URLs and apply
  platform-specific auth headers (GitLab, GitHub, Bitbucket)
- Web UI push/pull/update: use *WithAuth variants for git operations
- Export ResolveTokenForURL and DetectPlatformForURL from install/auth
2026-02-22 15:13:10 +08:00
Willie d1d528f192 feat: add --audit flag to hub index for security risk enrichment
Enrich hub index with per-skill audit scores so teammates can assess
skill safety before installing. Uses existing audit.ScanSkill() engine
(25 rules, 0-100 risk score) without modifying the audit package.

- SkillEntry gains riskScore (*int), riskLabel, auditedAt (omitempty)
- BuildIndex accepts variadic auditSkills param (backward compatible)
- search index deserializes and displays risk badges in hub results
- Schema stays v1; fields omitted when --audit not used
2026-02-22 08:08:35 +08:00
Willie 0b03a4e2fc fix(search): remove 20-result cap for hub search and add scroll-to-load
Hub search was capped at 20 results because it shared the same
normalizeLimit() path as GitHub search. Hub indexes are static JSON
(~100 entries), so returning all results is expected.

- Remove normalizeLimit() from hub search path; limit=0 means no limit
- Backend defaults to limit=0 for hub requests, 20 for GitHub
- CLI uses limit=0 when --hub is set without explicit --limit
- Frontend searchHub() no longer sends a limit parameter
- Replace IIFE rendering with useMemo for stable filter results
- Add IntersectionObserver-based scroll-to-load (20 items per batch)
2026-02-13 14:21:37 +08:00
Willie 6253099d7d fix: correct hub help text, gofmt alignment, and missing tags type
- Fix --index-url → --hub in hub index help text
- Fix gofmt struct field alignment in SearchResult after adding Tags
- Add missing tags field to frontend SearchResult TypeScript interface
2026-02-13 14:21:37 +08:00
Willie 88847ea3ae feat: add skill field to hub schema for multi-skill repo support
Hub JSON entries can now specify a `skill` field to target a specific
skill within a multi-skill repo. CLI auto-passes -s flag, Web UI
skips the picker modal when skill is specified.

Also adds Community Hub section to search docs with link to
runkids/skillshare-hub.
2026-02-13 14:21:37 +08:00
Willie 0aa60cd2e5 feat: add tags field to hub index and update docs
Add optional tags []string to SkillEntry and search index, enabling
classification and filtering of skills in hub catalogs. Tags are
read from SKILL.md frontmatter (comma-separated) during hub index
generation and included in search haystack for keyword matching.

Rewrite community-hub spec to reflect hand-curated hub.json model
(contributors edit JSON directly via PR, no skills/ directory).
Update website docs with tags schema, web dashboard screenshots,
and hub management instructions.
2026-02-13 14:21:37 +08:00
Willie 0ff683167a feat(search): add --index-url for private index search
SearchFromIndexURL consumes hub-generated index.json (local or HTTP),
with sourcePath resolution, relative path detection (including Windows),
and limit enforcement. The search CLI routes to index search when
--index-url is provided, with conditional spinner text and scoped
GitHub-specific error handling.
2026-02-13 14:21:37 +08:00
Willie d2e0065553 feat(search): add relevance scoring, parallel enrichment, and repo-scoped search
Search results were ranked only by GitHub stars, causing unrelated
high-star repos to outrank dedicated skill repos. Additionally,
owner/repo queries (e.g., "vercel-labs/skills") were treated as
plain text search instead of repo-scoped search, returning
completely wrong results.

Changes:
- Add multi-signal relevance scoring (name 0.45, description 0.25,
  stars 0.30) with log10-normalized star counts
- Detect owner/repo[/subdir] queries and use GitHub `repo:` qualifier
- Parallelize star and description enrichment (WaitGroup + semaphore,
  cap 10) — reduces wall time from 3-6s to ~200-600ms
- Extract both name and description from SKILL.md frontmatter
- Add Score field with json:"-" tag (hidden from CLI --json and API)
- Remove overly generous ContainsAny partial matching in name scoring
2026-02-09 22:18:05 +08:00
Willie 0c49860d63 feat: add project mode labels, privacy-first messaging, and UI polish
- Add (project) labels to status, sync, collect, target, and uninstall
  headers for clear mode distinction (matching list/target list pattern)
- Add ui.WithModeLabel() helper for shared functions that run in both modes
- Update uninstall confirmation prompt: "from the project?" in project mode
- Add "Privacy-first" to README Why section and website homepage
- Replace "Unified Source" card with "Privacy-First" card on website
- Unify Quick Actions card heights with h-full in DashboardPage
- Add frontend rebuild note to Docker sandbox docs
2026-02-09 21:08:03 +08:00
Willie e4565005b5 refactor: reduce cyclomatic complexity across CLI commands
Extract options structs and helper functions to reduce complexity:
- cmdInit: 49 → 5 (extract parseInitArgs, validateInitOptions, etc.)
- cmdUninstall: 39 → 12 (extract parseUninstallArgs, resolveUninstallTarget)
- cmdList: 33 → 12 (extract parseListArgs, buildSkillEntries)
- cmdInstall: 28 → 6 (extract parseInstallArgs, dispatchInstall)
- search.Search: 29 → ~5 (extract fetchCodeSearchResults, processSearchItems)
- main: 24 → 9 (command map pattern replaces switch)
- targetRemove: 21 → 7 (extract parseTargetRemoveArgs, backupTargets)
- updateAllTrackedRepos: 20 → ~10 (extract updateTrackedRepoQuick)
- cmdPush: 19 → 10 (extract parsePushArgs, checkGitRepo)

Overall: 19 functions > 15 reduced to 11 (42% improvement)
2026-01-31 19:53:28 +08:00
Willie Hung 1aab744b04 feat(search): add skillshare search command
- Search GitHub for skills containing SKILL.md files
- Sort results by repository stars (most popular first)
- Interactive selector with "Search again" option
- Auto-detect gh CLI token for authentication
- Filter out forked repositories
- Support --json, --list, --limit flags
2026-01-31 15:29:49 +08:00