Files
skillshare/internal/install/install_queries.go
T
a352b8d243 feat(install): pin skills to a tag or commit from their web URL (#293)
* feat(install): pin GitHub tree/blob URLs to their ref

Hub entries only carry a source URL, so a team hub had no way to pin a
third-party skill to a reviewed tag or commit. The parser already read
tree/<ref>/ and blob/<ref>/ URLs but discarded the ref, installing from
the default branch instead of the version the URL names.

Keep that ref as Source.Branch. An explicit --branch (CLI) or branch
(API body) still overrides it; the server handlers now only override
when a branch is given. Batch installs group by repo and ref so two
pinned versions of one repo get separate clones.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): describe pinning hub sources to a tag or commit

Explain that a GitHub tree/<ref>/ source URL pins the install, in both
the hub index guide and the install reference.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(install): pin GitLab/Bitbucket URLs and resolve refs containing "/"

Pinning should not depend on the host, so GitLab (-/tree/<ref>/) and
Bitbucket (src/<ref>/) web URLs now keep their ref too.

Taking the first path segment as the ref broke branch names containing
"/": tree/feature/x/skills/foo cloned branch "feature" and failed. When
the first segment is not a branch or tag on the remote, the longest
matching one is used instead. A ref the remote does not have fails
with a hint to use --branch, rather than silently installing the
default branch. A saved branch that covers more of the path (from
--branch or config) moves the subdir without asking the remote.

The grouped-install round-trip test pinned -/tree/main/ against a repo
on master; the ref used to be ignored, so its branch is now renamed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): cover GitLab, Bitbucket and "/" branches in URL pinning

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(install): keep web URL refs correct on reinstall, --branch and grouping

Review of the slash-branch resolution found four gaps:

- A saved branch equal to the whole URL tail (tree/feature/x) did not
  move the subdir back, so every reinstall or update looked for "x".
- An unrelated --branch with a slash-branch URL kept the parse-time
  subdir (x/skills/foo) instead of the URL's real one.
- Grouped installs copy the source to a repo root; resolution then put
  the subdir back and discovery looked under a doubled path. Grouping
  also keyed on the unresolved first segment, so tree/feature/x and
  tree/feature/y could share one clone. Web URLs with a ref now install
  on their own, and resolution leaves a repo-root copy at its root.
- A skill named after the parse-time subdir kept that name once the
  path turned out to be all ref (tree/feature/x named the skill "x").

The unknown-ref error now reads as a missing ref first, since a stale
ref (tree/master after a rename) is more common than a slash branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(update): keep legacy installs on the default branch

Skills installed before URL refs were honoured got the default branch
and recorded no branch. Single-skill update re-parses the recorded
source, so it moved them to the URL's ref (or failed on a stale one,
with a hint to pass --branch, which update does not accept), while
update --all rebuilds from RepoURL and kept the default branch.

ApplyRecordedBranch treats an empty recorded branch as the remote
default and drops the URL's ref. It replaces the branch handling at
every site that reinstalls from recorded metadata: CLI and server
update for skills and agents, and tracked-repo rehydration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* perf(search): share one clone across pinned hub skills from a repo

Any web URL with a path after its ref installed on its own, so a hub
listing many tree/<tag>/ skills from one repo cloned it once per skill.

groupByRepo now resolves each URL's ref before grouping, with a
RemoteRefs cache that lists each remote's refs once, and groups by
repo and resolved ref. Only refs containing "/" (whose subdir the
discovery match cannot re-derive) and refs that fail to resolve still
install alone. A resolved source is marked settled so the grouped
clone does not resolve it again, and ApplyRecordedBranch settles the
subdir from a recorded branch without asking the remote.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Willie <egg358358@gmail.com>
2026-09-28 22:08:16 +08:00

252 lines
6.8 KiB
Go

package install
import (
"fmt"
"os"
"path/filepath"
"strings"
"skillshare/internal/utils"
)
func getUpdatableSkillsImpl(sourceDir string) ([]string, error) {
store, err := LoadMetadata(sourceDir)
if err != nil {
return nil, err
}
var skills []string
for _, name := range store.List() {
entry := store.Get(name)
if entry == nil || entry.Source == "" {
continue
}
// Skip tracked repos (they are handled separately)
if entry.Tracked {
continue
}
skills = append(skills, KeyToRelPath(name, entry))
}
return skills, nil
}
// TrackedRepoMeta describes a tracked repository declared in metadata.
type TrackedRepoMeta struct {
Name string
Source string
Branch string
}
// getMissingTrackedReposImpl returns tracked repositories declared in .metadata.json
// whose clone directories are absent or no longer contain a git checkout.
func getMissingTrackedReposImpl(sourceDir string) ([]TrackedRepoMeta, error) {
store, err := LoadMetadata(sourceDir)
if err != nil {
return nil, err
}
existingRepos, err := GetTrackedRepos(sourceDir)
if err != nil {
return nil, err
}
existing := make(map[string]bool, len(existingRepos))
for _, repo := range existingRepos {
existing[filepath.ToSlash(repo)] = true
}
var missing []TrackedRepoMeta
for _, key := range store.List() {
entry := store.Get(key)
if entry == nil || !entry.Tracked {
continue
}
relPath := filepath.ToSlash(KeyToRelPath(key, entry))
if !strings.HasPrefix(filepath.Base(relPath), "_") {
continue
}
if existing[relPath] {
continue
}
missing = append(missing, TrackedRepoMeta{
Name: relPath,
Source: entry.Source,
Branch: entry.Branch,
})
}
return missing, nil
}
// RehydrateResult reports the outcome of rehydrating one missing tracked repo.
type RehydrateResult struct {
Name string `json:"name"`
Action string `json:"action"` // "rehydrated" | "error"
Error string `json:"error,omitempty"`
}
// rehydrateMissingTrackedReposImpl re-clones tracked repos declared in metadata
// whose clone directories are absent on disk. It mirrors the tracked-repo phase
// of InstallFromConfig (bare `skillshare install`); repos already on disk are
// left untouched. See issue #212.
func rehydrateMissingTrackedReposImpl(sourceDir string, parseOpts ParseOptions, opts InstallOptions) ([]RehydrateResult, error) {
store, err := LoadMetadata(sourceDir)
if err != nil {
return nil, err
}
existingRepos, err := GetTrackedRepos(sourceDir)
if err != nil {
return nil, err
}
existing := make(map[string]bool, len(existingRepos))
for _, repo := range existingRepos {
existing[filepath.ToSlash(repo)] = true
}
opts.Quiet = true
opts.Update = false
var results []RehydrateResult
for _, key := range store.List() {
entry := store.Get(key)
if entry == nil || !entry.Tracked {
continue
}
relPath := filepath.ToSlash(KeyToRelPath(key, entry))
if !strings.HasPrefix(filepath.Base(relPath), "_") {
continue
}
if existing[relPath] {
continue
}
groupDir, bareName := splitTrackedRelPath(relPath)
source, perr := ParseSourceWithOptions(entry.Source, parseOpts)
if perr != nil {
results = append(results, RehydrateResult{Name: relPath, Action: "error", Error: "invalid source: " + perr.Error()})
continue
}
source.Name = bareName
source.ApplyRecordedBranch(entry.Branch)
trackOpts := opts
trackOpts.Name = bareName
if groupDir != "" {
trackOpts.Into = groupDir
}
if _, ierr := InstallTrackedRepo(source, sourceDir, trackOpts); ierr != nil {
results = append(results, RehydrateResult{Name: relPath, Action: "error", Error: ierr.Error()})
continue
}
results = append(results, RehydrateResult{Name: relPath, Action: "rehydrated"})
}
return results, nil
}
// splitTrackedRelPath splits a tracked repo's relative path into its parent
// group directory and base name (the base retains its leading "_").
func splitTrackedRelPath(relPath string) (group, name string) {
relPath = strings.Trim(relPath, "/")
if idx := strings.LastIndex(relPath, "/"); idx >= 0 {
return relPath[:idx], relPath[idx+1:]
}
return "", relPath
}
// FindRepoInstalls scans sourceDir for skills whose meta repo_url matches
// cloneURL. Returns relative paths (e.g. "feature-radar/feature-radar-archive").
// Tracked repos (_-prefixed) are skipped.
func FindRepoInstalls(sourceDir, cloneURL string) []string {
if cloneURL == "" {
return nil
}
store, err := LoadMetadata(sourceDir)
if err != nil {
return nil
}
var matches []string
for _, name := range store.List() {
entry := store.Get(name)
if entry == nil || entry.Tracked {
continue
}
if repoURLsMatch(entry.RepoURL, cloneURL) {
matches = append(matches, KeyToRelPath(name, entry))
}
}
return matches
}
// CheckCrossPathDuplicate checks if a repo is already installed at a different
// location in sourceDir. Returns a user-facing error when duplicates are found
// outside targetPrefix, or nil if safe to proceed.
// Callers should skip this check when force is true or cloneURL is empty.
func CheckCrossPathDuplicate(sourceDir, cloneURL, targetPrefix string) error {
existing := FindRepoInstalls(sourceDir, cloneURL)
if len(existing) == 0 {
return nil
}
var elsewhere []string
for _, rel := range existing {
sameLocation := false
if targetPrefix == "" {
sameLocation = !strings.Contains(rel, "/")
} else {
sameLocation = rel == targetPrefix || strings.HasPrefix(rel, targetPrefix+"/")
}
if !sameLocation {
elsewhere = append(elsewhere, rel)
}
}
if len(elsewhere) == 0 {
return nil
}
loc := elsewhere[0]
if len(elsewhere) > 1 {
loc = fmt.Sprintf("%s (and %d more)", loc, len(elsewhere)-1)
}
return fmt.Errorf(
"this repo is already installed at skills/%s\n"+
"Use 'skillshare update' to refresh, or reinstall with --force to allow duplicates",
loc)
}
// getTrackedReposImpl returns tracked repositories from the source directory.
// It walks subdirectories recursively so repos nested in organizational
// directories (e.g. category/_team-repo/) are found.
func getTrackedReposImpl(sourceDir string) ([]string, error) {
var repos []string
walkRoot := utils.ResolveSymlink(sourceDir)
err := filepath.Walk(walkRoot, func(path string, info os.FileInfo, err error) error {
if err != nil {
return nil
}
if path == walkRoot {
return nil
}
// Skip .git directories
if info.IsDir() && info.Name() == ".git" {
return filepath.SkipDir
}
// Look for _-prefixed directories that are git repos
if info.IsDir() && len(info.Name()) > 0 && info.Name()[0] == '_' {
gitDir := filepath.Join(path, ".git")
if _, statErr := os.Stat(gitDir); statErr == nil {
relPath, relErr := filepath.Rel(walkRoot, path)
if relErr == nil {
repos = append(repos, relPath)
}
return filepath.SkipDir // Don't recurse into tracked repos
}
}
return nil
})
if err != nil {
return nil, err
}
return repos, nil
}