Files
skillshare/internal/server/handler_install.go
T
a352b8d243 feat(install): pin skills to a tag or commit from their web URL (#293)
* feat(install): pin GitHub tree/blob URLs to their ref

Hub entries only carry a source URL, so a team hub had no way to pin a
third-party skill to a reviewed tag or commit. The parser already read
tree/<ref>/ and blob/<ref>/ URLs but discarded the ref, installing from
the default branch instead of the version the URL names.

Keep that ref as Source.Branch. An explicit --branch (CLI) or branch
(API body) still overrides it; the server handlers now only override
when a branch is given. Batch installs group by repo and ref so two
pinned versions of one repo get separate clones.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): describe pinning hub sources to a tag or commit

Explain that a GitHub tree/<ref>/ source URL pins the install, in both
the hub index guide and the install reference.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(install): pin GitLab/Bitbucket URLs and resolve refs containing "/"

Pinning should not depend on the host, so GitLab (-/tree/<ref>/) and
Bitbucket (src/<ref>/) web URLs now keep their ref too.

Taking the first path segment as the ref broke branch names containing
"/": tree/feature/x/skills/foo cloned branch "feature" and failed. When
the first segment is not a branch or tag on the remote, the longest
matching one is used instead. A ref the remote does not have fails
with a hint to use --branch, rather than silently installing the
default branch. A saved branch that covers more of the path (from
--branch or config) moves the subdir without asking the remote.

The grouped-install round-trip test pinned -/tree/main/ against a repo
on master; the ref used to be ignored, so its branch is now renamed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): cover GitLab, Bitbucket and "/" branches in URL pinning

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(install): keep web URL refs correct on reinstall, --branch and grouping

Review of the slash-branch resolution found four gaps:

- A saved branch equal to the whole URL tail (tree/feature/x) did not
  move the subdir back, so every reinstall or update looked for "x".
- An unrelated --branch with a slash-branch URL kept the parse-time
  subdir (x/skills/foo) instead of the URL's real one.
- Grouped installs copy the source to a repo root; resolution then put
  the subdir back and discovery looked under a doubled path. Grouping
  also keyed on the unresolved first segment, so tree/feature/x and
  tree/feature/y could share one clone. Web URLs with a ref now install
  on their own, and resolution leaves a repo-root copy at its root.
- A skill named after the parse-time subdir kept that name once the
  path turned out to be all ref (tree/feature/x named the skill "x").

The unknown-ref error now reads as a missing ref first, since a stale
ref (tree/master after a rename) is more common than a slash branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(update): keep legacy installs on the default branch

Skills installed before URL refs were honoured got the default branch
and recorded no branch. Single-skill update re-parses the recorded
source, so it moved them to the URL's ref (or failed on a stale one,
with a hint to pass --branch, which update does not accept), while
update --all rebuilds from RepoURL and kept the default branch.

ApplyRecordedBranch treats an empty recorded branch as the remote
default and drops the URL's ref. It replaces the branch handling at
every site that reinstalls from recorded metadata: CLI and server
update for skills and agents, and tracked-repo rehydration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* perf(search): share one clone across pinned hub skills from a repo

Any web URL with a path after its ref installed on its own, so a hub
listing many tree/<tag>/ skills from one repo cloned it once per skill.

groupByRepo now resolves each URL's ref before grouping, with a
RemoteRefs cache that lists each remote's refs once, and groups by
repo and resolved ref. Only refs containing "/" (whose subdir the
discovery match cannot re-derive) and refs that fail to resolve still
install alone. A resolved source is marked settled so the grouped
clone does not resolve it again, and ApplyRecordedBranch settles the
subdir from a recorded branch without asking the remote.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Willie <egg358358@gmail.com>
2026-09-28 22:08:16 +08:00

556 lines
16 KiB
Go

package server
import (
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
"os"
"path/filepath"
"strconv"
"time"
"skillshare/internal/config"
"skillshare/internal/install"
)
// reloadSkillsStore refreshes the in-memory skills metadata store from disk.
// install.Install writes metadata to the file via its own store instance, so the
// server's cached s.skillsStore stays stale after an install — a freshly
// installed skill would then render with an empty source/type (shown as "Local"
// with a blank source in the UI) until the server restarts.
func (s *Server) reloadSkillsStore() {
if st, err := install.LoadMetadataWithMigration(s.cfg.EffectiveSkillsSource(), ""); err == nil && st != nil {
s.skillsStore = st
}
}
func discoverInstallSource(source *install.Source) (*install.DiscoveryResult, error) {
if source.IsGit() {
if source.HasSubdir() {
return install.DiscoverFromGitSubdir(source)
}
return install.DiscoverFromGit(source)
}
return install.DiscoverLocal(source)
}
// handleDiscover clones a git repo to a temp dir, discovers skills, then cleans up.
// Returns whether the caller needs to present a selection UI.
func (s *Server) handleDiscover(w http.ResponseWriter, r *http.Request) {
// Snapshot config under RLock, then release before I/O.
s.mu.RLock()
parseOpts := s.parseOpts()
s.mu.RUnlock()
var body struct {
Source string `json:"source"`
Branch string `json:"branch"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid JSON body")
return
}
if body.Source == "" {
writeError(w, http.StatusBadRequest, "source is required")
return
}
source, err := install.ParseSourceWithOptions(body.Source, parseOpts)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid source: "+err.Error())
return
}
if body.Branch != "" {
source.Branch = body.Branch
}
discovery, err := discoverInstallSource(source)
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
defer install.CleanupDiscovery(discovery)
skills := make([]map[string]string, len(discovery.Skills))
for i, sk := range discovery.Skills {
skills[i] = map[string]string{"name": sk.Name, "path": sk.Path, "description": sk.Description, "kind": "skill"}
}
agents := make([]map[string]string, len(discovery.Agents))
for i, ag := range discovery.Agents {
agents[i] = map[string]string{"name": ag.Name, "path": ag.Path, "fileName": ag.FileName, "kind": "agent"}
}
writeJSON(w, map[string]any{
"needsSelection": len(discovery.Skills) > 1 || len(discovery.Agents) > 0,
"skills": skills,
"agents": agents,
})
}
// handleInstallBatch re-clones a repo and installs each selected skill.
func (s *Server) handleInstallBatch(w http.ResponseWriter, r *http.Request) {
start := time.Now()
s.mu.Lock()
defer s.mu.Unlock()
defer s.trackProjectLock()()
var body struct {
Source string `json:"source"`
Branch string `json:"branch"`
Skills []struct {
Name string `json:"name"`
Path string `json:"path"`
} `json:"skills"`
Force bool `json:"force"`
SkipAudit bool `json:"skipAudit"`
Into string `json:"into"`
Name string `json:"name"`
Kind string `json:"kind,omitempty"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid JSON body")
return
}
if body.Source == "" || len(body.Skills) == 0 {
writeError(w, http.StatusBadRequest, "source and skills are required")
return
}
source, err := install.ParseSourceWithOptions(body.Source, s.parseOpts())
if err != nil {
writeError(w, http.StatusBadRequest, "invalid source: "+err.Error())
return
}
if body.Branch != "" {
source.Branch = body.Branch
}
if s.IsProjectMode() {
if err := install.RejectProjectRootLocalInstall(source, s.projectRoot); err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
}
discovery, err := discoverInstallSource(source)
if err != nil {
writeError(w, http.StatusInternalServerError, "discovery failed: "+err.Error())
return
}
defer install.CleanupDiscovery(discovery)
type batchResultItem struct {
Name string `json:"name"`
Action string `json:"action,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Error string `json:"error,omitempty"`
}
// Ensure Into directory exists
if body.Into != "" {
baseDir := s.cfg.EffectiveSkillsSource()
if body.Kind == "agent" {
baseDir = s.agentsSource()
}
if err := os.MkdirAll(filepath.Join(baseDir, body.Into), 0755); err != nil {
writeError(w, http.StatusInternalServerError, "failed to create into directory: "+err.Error())
return
}
}
// Note: no cross-path duplicate check for batch — user explicitly selected
// specific skills from discovery, so installing to a different path is intentional.
results := make([]batchResultItem, 0, len(body.Skills))
installOpts := install.InstallOptions{
Force: body.Force,
AuditOverride: body.Force,
SkipAudit: body.SkipAudit,
AuditThreshold: s.auditThreshold(),
Branch: body.Branch,
SourceDir: s.cfg.EffectiveSkillsSource(),
}
if s.IsProjectMode() {
installOpts.AuditProjectRoot = s.projectRoot
}
isAgent := body.Kind == "agent"
if isAgent {
installOpts.SourceDir = s.agentsSource()
}
for _, sel := range body.Skills {
skillName := sel.Name
if body.Name != "" && len(body.Skills) == 1 {
skillName = body.Name
}
if isAgent {
// Agent install: copy single .md file to agents source
agentsDir := s.agentsSource()
if body.Into != "" {
agentsDir = filepath.Join(agentsDir, body.Into)
}
agentInfo := install.AgentInfo{
Name: sel.Name,
Path: sel.Path,
FileName: filepath.Base(sel.Path),
}
res, err := install.InstallAgentFromDiscovery(discovery, agentInfo, agentsDir, installOpts)
if err != nil {
results = append(results, batchResultItem{
Name: skillName,
Error: err.Error(),
})
continue
}
results = append(results, batchResultItem{
Name: skillName,
Action: res.Action,
Warnings: res.Warnings,
})
} else {
// Skill install: copy directory to skills source
destPath := filepath.Join(s.cfg.EffectiveSkillsSource(), body.Into, skillName)
res, err := install.InstallFromDiscovery(discovery, install.SkillInfo{
Name: sel.Name,
Path: sel.Path,
}, destPath, installOpts)
if err != nil {
results = append(results, batchResultItem{
Name: skillName,
Error: err.Error(),
})
continue
}
results = append(results, batchResultItem{
Name: skillName,
Action: res.Action,
Warnings: res.Warnings,
})
}
}
// Summary for toast
installed := 0
installedSkills := make([]string, 0, len(results))
failedSkills := make([]string, 0, len(results))
var firstErr string
for _, r := range results {
if r.Error == "" {
installed++
installedSkills = append(installedSkills, r.Name)
} else if firstErr == "" {
firstErr = r.Error
failedSkills = append(failedSkills, r.Name)
} else {
failedSkills = append(failedSkills, r.Name)
}
}
kindLabel := "skills"
if isAgent {
kindLabel = "agents"
}
summary := fmt.Sprintf("Installed %d of %d %s", installed, len(body.Skills), kindLabel)
if firstErr != "" {
summary += " (some errors)"
}
if isAgent && installed > 0 && !s.cfg.HasAgentTarget() {
summary += ". Warning: none of your configured targets support agents"
}
status := "ok"
if installed < len(body.Skills) {
status = "partial"
}
args := map[string]any{
"source": body.Source,
"mode": s.installLogMode(),
"force": body.Force,
"scope": "ui",
"threshold": s.auditThreshold(),
"skill_count": installed,
}
if body.SkipAudit {
args["skip_audit"] = true
}
if body.Into != "" {
args["into"] = body.Into
}
if len(installedSkills) > 0 {
args["installed_skills"] = installedSkills
}
if len(failedSkills) > 0 {
args["failed_skills"] = failedSkills
}
s.writeOpsLog("install", status, start, args, firstErr)
// Reconcile config after install
if installed > 0 {
if isAgent {
if st, loadErr := install.LoadMetadataWithMigration(s.agentsSource(), install.MetadataKindAgent); loadErr == nil && st != nil {
s.agentsStore = st
}
} else {
s.reloadSkillsStore()
}
if s.IsProjectMode() {
if rErr := config.ReconcileProjectSkills(s.projectRoot, s.projectCfg, s.skillsStore, s.cfg.EffectiveSkillsSource()); rErr != nil {
log.Printf("warning: failed to reconcile project skills config: %v", rErr)
}
} else {
if rErr := config.ReconcileGlobalSkills(s.cfg, s.skillsStore); rErr != nil {
log.Printf("warning: failed to reconcile global skills config: %v", rErr)
}
}
}
writeJSON(w, map[string]any{
"results": results,
"summary": summary,
})
}
func (s *Server) handleInstall(w http.ResponseWriter, r *http.Request) {
start := time.Now()
s.mu.Lock()
defer s.mu.Unlock()
defer s.trackProjectLock()()
var body struct {
Source string `json:"source"`
Branch string `json:"branch"`
Name string `json:"name"`
Force bool `json:"force"`
SkipAudit bool `json:"skipAudit"`
Track bool `json:"track"`
Into string `json:"into"`
Kind string `json:"kind,omitempty"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid JSON body")
return
}
if body.Source == "" {
writeError(w, http.StatusBadRequest, "source is required")
return
}
source, err := install.ParseSourceWithOptions(body.Source, s.parseOpts())
if err != nil {
writeError(w, http.StatusBadRequest, "invalid source: "+err.Error())
return
}
if body.Branch != "" {
source.Branch = body.Branch
}
if s.IsProjectMode() {
if err := install.RejectProjectRootLocalInstall(source, s.projectRoot); err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
}
if body.Name != "" {
source.Name = body.Name
}
// Tracked repo install
if body.Track {
trackedKind, err := install.InferTrackedKind(source, body.Kind)
if err != nil {
var ambig *install.TrackKindAmbiguousError
if errors.As(err, &ambig) {
writeCodedError(w, http.StatusBadRequest,
"install.track_kind_ambiguous", err.Error(),
map[string]string{
"skills": strconv.Itoa(ambig.Skills),
"agents": strconv.Itoa(ambig.Agents),
})
return
}
writeError(w, http.StatusBadRequest, err.Error())
return
}
trackSourceDir := s.cfg.EffectiveSkillsSource()
if trackedKind == "agent" {
trackSourceDir = s.agentsSource()
}
installOpts := install.InstallOptions{
Name: body.Name,
Kind: trackedKind,
Force: body.Force,
AuditOverride: body.Force,
SkipAudit: body.SkipAudit,
Into: body.Into,
Branch: body.Branch,
AuditThreshold: s.auditThreshold(),
SourceDir: trackSourceDir,
}
if s.IsProjectMode() {
installOpts.AuditProjectRoot = s.projectRoot
}
result, err := install.InstallTrackedRepo(source, trackSourceDir, install.InstallOptions{
Name: installOpts.Name,
Kind: installOpts.Kind,
Force: installOpts.Force,
AuditOverride: installOpts.AuditOverride,
SkipAudit: installOpts.SkipAudit,
Into: installOpts.Into,
Branch: installOpts.Branch,
AuditThreshold: installOpts.AuditThreshold,
AuditProjectRoot: installOpts.AuditProjectRoot,
})
if err != nil {
s.writeOpsLog("install", "error", start, map[string]any{
"source": body.Source,
"mode": s.installLogMode(),
"tracked": true,
"force": body.Force,
"threshold": s.auditThreshold(),
"scope": "ui",
"failed_skills": []string{source.Name},
}, err.Error())
writeError(w, http.StatusInternalServerError, err.Error())
return
}
// Reconcile config after tracked repo install
if trackedKind == "skill" {
s.reloadSkillsStore()
if s.IsProjectMode() {
if rErr := config.ReconcileProjectSkills(s.projectRoot, s.projectCfg, s.skillsStore, s.cfg.EffectiveSkillsSource()); rErr != nil {
log.Printf("warning: failed to reconcile project skills config: %v", rErr)
}
} else {
if rErr := config.ReconcileGlobalSkills(s.cfg, s.skillsStore); rErr != nil {
log.Printf("warning: failed to reconcile global skills config: %v", rErr)
}
}
}
args := map[string]any{
"source": body.Source,
"mode": s.installLogMode(),
"tracked": true,
"kind": trackedKind,
"force": body.Force,
"threshold": s.auditThreshold(),
"scope": "ui",
"skill_count": result.SkillCount,
}
if body.SkipAudit {
args["skip_audit"] = true
}
if body.Into != "" {
args["into"] = body.Into
}
if len(result.Skills) > 0 {
args["installed_skills"] = result.Skills
}
s.writeOpsLog("install", "ok", start, args, "")
writeJSON(w, map[string]any{
"repoName": result.RepoName,
"skillCount": result.SkillCount,
"agentCount": result.AgentCount,
"skills": result.Skills,
"agents": result.Agents,
"action": result.Action,
"warnings": result.Warnings,
})
return
}
// Cross-path duplicate detection
if !body.Force && source.CloneURL != "" {
if err := install.CheckCrossPathDuplicate(s.cfg.EffectiveSkillsSource(), source.CloneURL, body.Into); err != nil {
writeError(w, http.StatusConflict, err.Error())
return
}
}
// Regular install
destPath := filepath.Join(s.cfg.EffectiveSkillsSource(), body.Into, source.Name)
if body.Into != "" {
if err := os.MkdirAll(filepath.Join(s.cfg.EffectiveSkillsSource(), body.Into), 0755); err != nil {
writeError(w, http.StatusInternalServerError, "failed to create into directory: "+err.Error())
return
}
}
result, err := install.Install(source, destPath, install.InstallOptions{
Name: body.Name,
Force: body.Force,
AuditOverride: body.Force,
SkipAudit: body.SkipAudit,
Branch: body.Branch,
AuditThreshold: s.auditThreshold(),
AuditProjectRoot: func() string {
if s.IsProjectMode() {
return s.projectRoot
}
return ""
}(),
})
if err != nil {
s.writeOpsLog("install", "error", start, map[string]any{
"source": body.Source,
"mode": s.installLogMode(),
"force": body.Force,
"threshold": s.auditThreshold(),
"scope": "ui",
"failed_skills": []string{source.Name},
}, err.Error())
writeError(w, http.StatusInternalServerError, err.Error())
return
}
// Reconcile config after single install
s.reloadSkillsStore()
if s.IsProjectMode() {
if rErr := config.ReconcileProjectSkills(s.projectRoot, s.projectCfg, s.skillsStore, s.cfg.EffectiveSkillsSource()); rErr != nil {
log.Printf("warning: failed to reconcile project skills config: %v", rErr)
}
} else {
if rErr := config.ReconcileGlobalSkills(s.cfg, s.skillsStore); rErr != nil {
log.Printf("warning: failed to reconcile global skills config: %v", rErr)
}
}
okArgs := map[string]any{
"source": body.Source,
"mode": s.installLogMode(),
"force": body.Force,
"threshold": s.auditThreshold(),
"scope": "ui",
"skill_count": 1,
"installed_skills": []string{result.SkillName},
}
if body.SkipAudit {
okArgs["skip_audit"] = true
}
if body.Into != "" {
okArgs["into"] = body.Into
}
s.writeOpsLog("install", "ok", start, okArgs, "")
writeJSON(w, map[string]any{
"skillName": result.SkillName,
"action": result.Action,
"warnings": result.Warnings,
})
}
func (s *Server) installLogMode() string {
if s.IsProjectMode() {
return "project"
}
return "global"
}