Files
skillshare/internal/install/install_config.go
T
Willie dcffa2ec5e fix(install): keep a moved skill's record instead of reinstalling it (#513)
* fix(install): keep a moved skill's record instead of reinstalling it

Install records are keyed by path, so a skill moved with mv shows up as
missing. "Install missing" (and bare `skillshare install`) then cloned it
again at the old path, leaving two copies that both sync to targets, and
reconcile could hand the new copy's record to the moved one.

A gone record now follows an unrecorded directory of the same name whose
files match the recorded hashes, when exactly one such directory exists.
Install-from-config skips such entries and reconcile re-keys the record,
so the moved copy keeps its source. Same-name skills with other files are
left alone, so an update can never overwrite them from that source.

Refs: #510

* fix(install): keep audit acceptances and unreadable links out of moves

Re-keying a moved skill went through Remove, which also drops the audit
findings accepted for the old path, so an unchanged skill could block or
prompt again on its next update. MoveEntry now carries them to the new key.

A record below a first-level source link that cannot be read looks gone
to Lstat, so an identical copy elsewhere could take its record while the
link was down. Such records are no longer matched as moved.

* fix(install): drop legacy-path acceptances and respect lock pins on move

A legacy basename key such as `demo` with group `old` keeps its accepted
audit findings under `old/demo`. MoveEntry removed only the key's own
acceptances, so the old path kept them and a later skill placed there
would inherit them without a new --force. It now deletes the old path.

When the lockfile pins another commit than the moved copy holds, treating
the copy as the moved skill re-keyed its stale record and let the project
lock be rewritten with the older commit. Such copies no longer count as a
move, so the pinned commit is installed as before.

* fix(install): search for moved skills only where reconcile looks

The search for a moved copy walked into installed skills and tracked
checkouts, which reconcile skips. A copy found there made install skip the
skill, then reconcile never saw the copy and pruned the record, and in
project mode the config entry with it. The search now stops at the same
boundaries.

* fix(install): never take a recorded skill as another's moved copy

Two identical skills can be recorded at different paths. When one of them
went missing, the other matched its hashes and was taken as its moved
copy, so install skipped the missing skill and reconcile pruned its record.
A directory with a record of its own is now never a move destination.

* fix(install): derive moved-copy paths from the resolved source root

sourcewalk.WalkDir reports paths under the resolved source root, but the
moved-copy search took them relative to the unresolved one. With a linked
source (or Windows short temp paths) every path came out as ../..., so the
installed-skill boundary never matched and a copy inside an installed
parent could be taken as the move, after which reconcile pruned the record.
The search now resolves the root first, as reconcile does.

* fix(install): move a record only to one copy of what the config declares

Three cases still let a move drop or misplace a record:

- In project mode the manifest can point a skill at a new source while a
  copy of the old one exists elsewhere; that copy was taken as the skill,
  so the new source was never installed. The record's source and branch
  must now match the config entry.
- A git checkout reconcile treats as tracked could be accepted before the
  tracked boundary was checked; the boundaries now run first.
- With two identical copies, reconcile re-keyed the record to whichever
  it walked first. Both reconcile and the install-from-config search now
  require exactly one destination, and reconcile decides after the walk.

* fix(install): clear accepted findings left at a move destination

A removed legacy grouped entry can leave findings accepted for its path
behind. MoveEntry only wrote the destination when the moved skill had its
own acceptances, so a skill moved into that path inherited findings it
never accepted. The destination is now cleared before the transfer.

* fix(install): follow a moved skill only while its declaration still matches

The install-from-config search compared source and branch but not the
tracking mode, so a skill switched to tracked was satisfied by an old
plain copy and never cloned.

Project reconciliation, which other installs also run, inferred moves
without looking at the manifest at all. A record whose declaration now
names another source could follow a copy of the old source, after which
the manifest entry was rewritten to that old source. Moves in project
mode now require the declaration to still match the record, through the
same MatchesDeclaration check the install path uses.

* fix(install): infer no move while a source link is unreadable

An unreadable followed source link can hide a second identical copy, so
a copy that looked unique could take the record arbitrarily. Reconcile
and the install-from-config search now infer a move only from a complete
inventory; reconcile already keeps the gone record in that case.

* fix(install): infer no move when the source walk hit an error

Both walks dropped traversal errors, so an unreadable directory that hid
a second identical copy left the visible one looking unique and it could
take the record. Any walk error now counts as an incomplete inventory,
like an unreadable source link, and no move is inferred from it.

* fix(install): keep a record whose move waits on a failed walk

When a walk error deferred a move, the walk still counted as complete, so
pruning dropped the gone record along with its accepted audit findings
and it could never follow the copy. A failed walk that deferred a move
now marks the inventory incomplete, as an unreadable source link does;
a failed walk with no move candidate keeps the old behavior.

Also correct InstallFromConfig's comments, which still said reconcile
runs only after an install.

* fix(install): move a moved skill's gitignore rule in project mode

Project reconcile added the managed ignore rule for a moved skill's new
path but left the old one, and the gitignore update only ever adds. If
the old path later held source-controlled files, Git kept ignoring them
without a word. Reconcile now reports the paths records moved away from,
and project reconcile removes their managed rules before adding the new.

* test(config): skip the unreadable-directory reconcile test on Windows

Windows does not stop reading a directory created with mode 0, and
os.Geteuid reports -1 there, so the root-only skip let the test run and
fail on the Windows runner.

* fix(install): count a candidate that cannot be hashed as an incomplete search

MovedEntryKey dropped a same-name candidate whose files could not be
hashed, so a readable identical copy beside it looked unique and could
take the record. It now returns the hash error, and reconcile and the
install-from-config search treat it like a walk error: no move is
inferred from that scan, and reconcile keeps the record waiting.
2026-10-10 23:14:52 +08:00

551 lines
16 KiB
Go

package install
import (
"fmt"
"os"
"path"
"path/filepath"
"strings"
"skillshare/internal/sourcefs"
"skillshare/internal/sourcewalk"
"skillshare/internal/ui"
"skillshare/internal/utils"
"skillshare/internal/validate"
)
// configSkillEntry pairs a SkillEntryDTO with its parsed Source.
type configSkillEntry struct {
dto SkillEntryDTO
source *Source
relock bool // already installed, but at another commit than the lockfile's
}
// optsFor stages a relock like an update, so a blocked audit keeps the
// installed copy.
func (e configSkillEntry) optsFor(opts InstallOptions) InstallOptions {
if e.relock {
opts.Force, opts.Update = true, true
}
return opts
}
// configSkillGroup holds config skills sharing the same CloneURL.
// The group is cloned once; each skill is then installed from the local clone.
type configSkillGroup struct {
cloneURL string
skills []configSkillEntry
}
// groupConfigSkillsByRepo partitions parsed config entries by CloneURL + Branch.
// Entries that share the same git repo AND branch (2+ skills with Subdir) are grouped
// for a single clone. Entries that cannot be grouped (local path, no subdir,
// or sole skill from a repo) are returned in the singles slice.
func groupConfigSkillsByRepo(entries []configSkillEntry) (groups []configSkillGroup, singles []configSkillEntry) {
buckets := make(map[string]*configSkillGroup)
var order []string
for _, e := range entries {
if !e.source.IsGit() || e.source.Subdir == "" || e.source.HasAmbiguousWebRef() {
singles = append(singles, e)
continue
}
// Include branch in key so same repo on different branches are not grouped together
key := e.source.CloneURL
if e.source.Branch != "" {
key += "\t" + e.source.Branch
}
// Skills of one repo installed at different times are locked apart.
key += "\t" + e.source.Commit
if g, ok := buckets[key]; ok {
g.skills = append(g.skills, e)
} else {
buckets[key] = &configSkillGroup{
cloneURL: e.source.CloneURL,
skills: []configSkillEntry{e},
}
order = append(order, key)
}
}
for _, key := range order {
g := buckets[key]
if len(g.skills) < 2 {
singles = append(singles, g.skills[0])
} else {
groups = append(groups, *g)
}
}
return groups, singles
}
// repoSourceForConfigGroup converts a subdir source into a repo-root source
// for whole-repo cloning. Similar to repoSourceForGroupedClone in search_batch.go.
func repoSourceForConfigGroup(src *Source) Source {
repoSource := *src
repoSource.Subdir = ""
repoSource.Raw = repoSource.CloneURL
if root, err := ParseSource(repoSource.CloneURL); err == nil {
repoSource.Type = root.Type
repoSource.Raw = root.Raw
repoSource.Name = root.Name
}
return repoSource
}
// matchDiscoveredSkillBySubdir finds a SkillInfo in the discovery result that
// matches the given subdir path.
func matchDiscoveredSkillBySubdir(discovery *DiscoveryResult, subdir string) (SkillInfo, bool) {
for _, skill := range discovery.Skills {
if skill.Path == subdir {
return skill, true
}
}
return SkillInfo{}, false
}
// MetadataSkillEntries lists the entries of a metadata store as config skills,
// sorted by key; it is the global-mode source of InstallContext.ConfigSkills.
func MetadataSkillEntries(store *MetadataStore) []SkillEntryDTO {
names := store.List() // sorted
dtos := make([]SkillEntryDTO, 0, len(names))
for _, name := range names {
entry := store.Get(name)
if entry == nil {
continue
}
group, bareName := splitTrackedRelPath(filepath.ToSlash(KeyToRelPath(name, entry)))
dtos = append(dtos, SkillEntryDTO{
Name: bareName,
Source: entry.Source,
Tracked: entry.Tracked,
Group: group,
Branch: entry.Branch,
})
}
return dtos
}
// MissingFromConfig returns the config skills InstallFromConfig would install:
// entries with a name whose directory does not exist on disk.
func MissingFromConfig(ctx InstallContext) []SkillEntryDTO {
sourcePath := ctx.SourcePath()
var missing []SkillEntryDTO
for _, skill := range ctx.ConfigSkills() {
if _, bareName := skill.EffectiveParts(); strings.TrimSpace(bareName) == "" {
continue
}
if _, err := os.Stat(filepath.Join(sourcePath, filepath.FromSlash(skill.FullName()))); err != nil {
missing = append(missing, skill)
}
}
return missing
}
// InstallFromConfig iterates over the remote skills listed in the config
// (via ctx.ConfigSkills) and installs each one that is not already present.
// It handles both tracked repos and plain skills, delegates per-skill hooks
// to ctx.PostInstallSkill, and calls ctx.Reconcile when at least one skill
// was installed or found moved, so the moved skill's record follows it.
//
// The caller is responsible for UI chrome (logo, spinner, next-steps).
func InstallFromConfig(ctx InstallContext, opts InstallOptions) (ConfigInstallResult, error) {
result := ConfigInstallResult{
InstalledSkills: make([]string, 0),
FailedSkills: make([]string, 0),
}
sourcePath := ctx.SourcePath()
// A grouped entry installs to sourcePath/<group>/<name> without opts.Into, so
// the skills root cannot be derived from the destination path.
opts.SourceDir = sourcePath
parseOpts := ParseOptions{
GitLabHosts: ctx.GitLabHosts(),
AzureHosts: ctx.AzureHosts(),
CNBHosts: ctx.CNBHosts(),
GiteaHosts: ctx.GiteaHosts(),
}
// ── Classify skills: tracked / plain (groupable vs singles) ──
var tracked []SkillEntryDTO
var plain []configSkillEntry
store := LoadMetadataOrNew(sourcePath)
moved := 0
for _, skill := range ctx.ConfigSkills() {
_, bareName := skill.EffectiveParts()
if strings.TrimSpace(bareName) == "" {
continue
}
displayName := skill.FullName()
destPath := filepath.Join(sourcePath, filepath.FromSlash(displayName))
locked := opts.Lock.CommitFor(displayName, skill.Source)
// Skip skills that already exist on disk, unless the lockfile moved on
// (a teammate updated) and this copy has to follow.
relock := false
if _, err := os.Stat(destPath); err == nil {
if locked == "" || InstalledCommit(destPath, store.GetByPath(displayName)) == locked {
result.Skipped++
if skill.Tracked {
result.SkippedRepos++
}
if !opts.Quiet {
ui.StepDone(displayName, "skipped (already exists)")
}
continue
}
if opts.DryRun {
if !opts.Quiet {
ui.StepDone(displayName, "would move to locked commit "+shortHash(locked))
}
continue
}
if IsGitRepo(destPath) {
if err := relockGitRepo(destPath, locked, opts); err != nil {
if !opts.Quiet {
ui.StepFail(displayName, err.Error())
}
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
if !opts.Quiet {
ui.StepDone(displayName, "moved to locked commit "+shortHash(locked))
}
result.InstalledSkills = append(result.InstalledSkills, displayName)
result.Installed++
continue
}
relock = true
} else if to := movedCopyOf(store, sourcePath, skill, opts.SourceFollow, locked); to != "" {
// Reconcile moves the record to the copy; installing would duplicate it.
result.Skipped++
if !opts.DryRun {
moved++
}
if !opts.Quiet {
ui.StepDone(displayName, "moved to "+to+", record follows")
}
continue
}
source, err := ParseSourceWithOptions(skill.Source, parseOpts)
if err != nil {
if !opts.Quiet {
ui.StepFail(displayName, fmt.Sprintf("invalid source: %v", err))
}
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
source.Name = bareName
source.ApplyRecordedBranch(skill.Branch)
source.Commit = locked
if skill.Tracked {
tracked = append(tracked, skill)
} else {
plain = append(plain, configSkillEntry{dto: skill, source: source, relock: relock})
}
}
groups, singles := groupConfigSkillsByRepo(plain)
// ── Phase 1: tracked repos (unchanged — each does its own clone+discover) ──
for _, skill := range tracked {
groupDir, bareName := skill.EffectiveParts()
displayName := skill.FullName()
source, err := ParseSourceWithOptions(skill.Source, parseOpts)
if err != nil {
if !opts.Quiet {
ui.StepFail(displayName, fmt.Sprintf("invalid source: %v", err))
}
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
source.Name = bareName
source.Branch = skill.Branch
source.Commit = opts.Lock.CommitFor(displayName, skill.Source)
installed := installTrackedFromConfig(source, sourcePath, displayName, groupDir, opts)
if installed.failed {
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
if opts.DryRun {
if !opts.Quiet {
ui.StepDone(displayName, installed.action)
}
continue
}
result.InstalledSkills = append(result.InstalledSkills, installed.skills...)
result.Installed++
result.InstalledRepos++
result.InstalledRepoSkills += installed.skillCount
}
// ── Phase 2: grouped plain skills (clone once per repo) ──
for _, group := range groups {
repoSource := repoSourceForConfigGroup(group.skills[0].source)
discovery, cloneErr := DiscoverFromGitWithProgress(&repoSource, opts.OnProgress)
if cloneErr != nil {
// Clone failed — mark all skills in group as failed.
for _, e := range group.skills {
displayName := e.dto.FullName()
if !opts.Quiet {
ui.StepFail(displayName, fmt.Sprintf("clone failed: %v", cloneErr))
}
result.FailedSkills = append(result.FailedSkills, displayName)
}
continue
}
var fallback []configSkillEntry
for _, e := range group.skills {
groupDir, bareName := e.dto.EffectiveParts()
displayName := e.dto.FullName()
destPath := filepath.Join(sourcePath, filepath.FromSlash(displayName))
skill, found := matchDiscoveredSkillBySubdir(discovery, e.source.Subdir)
if !found {
// Can't match — fall back to Phase 3 individual install.
fallback = append(fallback, e)
continue
}
if err := validate.SkillName(bareName); err != nil {
if !opts.Quiet {
ui.StepFail(displayName, fmt.Sprintf("invalid name: %v", err))
}
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
if groupDir != "" {
if err := sourcefs.MkdirAllIn(sourcePath, filepath.FromSlash(groupDir), opts.SourceFollow); err != nil {
if !opts.Quiet {
ui.StepFail(displayName, fmt.Sprintf("failed to create group directory: %v", err))
}
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
}
_, err := InstallFromDiscovery(discovery, skill, destPath, e.optsFor(opts))
if err != nil {
if !opts.Quiet {
ui.StepFail(displayName, err.Error())
}
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
if opts.DryRun {
if !opts.Quiet {
ui.StepDone(displayName, "would install (grouped)")
}
continue
}
if err := ctx.PostInstallSkill(displayName); err != nil {
ui.Warning("post-install hook failed for %s: %v", displayName, err)
}
if !opts.Quiet {
ui.StepDone(displayName, "installed")
}
result.InstalledSkills = append(result.InstalledSkills, displayName)
result.Installed++
}
CleanupDiscovery(discovery)
singles = append(singles, fallback...)
}
// ── Phase 3: singles (unchanged per-skill flow) ──
for _, e := range singles {
groupDir, bareName := e.dto.EffectiveParts()
displayName := e.dto.FullName()
destPath := filepath.Join(sourcePath, filepath.FromSlash(displayName))
ok := installPlainFromConfig(ctx, e.source, sourcePath, destPath, displayName, bareName, groupDir, e.optsFor(opts))
if !ok {
result.FailedSkills = append(result.FailedSkills, displayName)
continue
}
if opts.DryRun {
continue // StepDone already called inside installPlainFromConfig
}
result.InstalledSkills = append(result.InstalledSkills, displayName)
result.Installed++
}
// ── Phase 4: Reconcile config after installs and moves ──
if (result.Installed > 0 || moved > 0) && !opts.DryRun {
if err := ctx.Reconcile(); err != nil {
return result, err
}
}
return result, nil
}
// movedCopyOf returns the source-relative path the recorded skill was moved
// to (see MetadataStore.MovedEntryKey), or "" unless exactly one copy matches.
// The record must still install what skill declares, and a copy at another
// commit than locked does not count, so the lockfile's pin is kept.
func movedCopyOf(store *MetadataStore, sourcePath string, skill SkillEntryDTO, follow *sourcewalk.Follow, locked string) string {
displayName := skill.FullName()
entry := store.GetByPath(displayName)
if entry == nil || len(entry.FileHashes) == 0 || !entry.MatchesDeclaration(skill) {
return ""
}
var found []string
walkFailed := false // an unreadable directory may hide a copy
// The walk reports paths under the resolved root, as in reconcile.
root := utils.ResolveSymlink(sourcePath)
_ = sourcewalk.WalkDir(root, sourcewalk.Options{Follow: follow}, func(p string, d os.DirEntry, err error) error {
if err != nil {
walkFailed = true
return nil
}
if !d.IsDir() || p == root {
return nil
}
if utils.IsHidden(d.Name()) {
return filepath.SkipDir
}
rel, relErr := filepath.Rel(root, p)
if relErr != nil {
return nil
}
// Reconcile does not look inside an installed skill or tracked checkout,
// so a copy there could not take the record.
if e := store.GetByPath(rel); IsTrackedCheckout(p) || e != nil && e != entry && e.Source != "" {
return filepath.SkipDir
}
if d.Name() == path.Base(displayName) {
key, hashErr := store.MovedEntryKey(root, rel, p, follow)
if hashErr != nil {
walkFailed = true
}
if key != "" && store.Entries[key] == entry {
found = append(found, filepath.ToSlash(rel))
return filepath.SkipDir
}
}
return nil
})
// An unreadable source link may hide another copy.
if len(found) != 1 || walkFailed || follow.Incomplete() || locked != "" && InstalledCommit(filepath.Join(root, filepath.FromSlash(found[0])), entry) != locked {
return ""
}
return found[0]
}
// trackedInstallOutcome captures the result of a single tracked-repo install
// within the config loop.
type trackedInstallOutcome struct {
failed bool
action string // only meaningful on dry-run
skills []string // skills names to record
skillCount int // skills discovered in the repo
}
// installTrackedFromConfig installs a single tracked repo from config and
// returns a summary used by the caller loop.
func installTrackedFromConfig(
source *Source,
sourcePath string,
displayName string,
groupDir string,
opts InstallOptions,
) trackedInstallOutcome {
trackOpts := opts
trackOpts.Name = source.Name // keep the name recorded in config
if groupDir != "" {
trackOpts.Into = groupDir
}
trackedResult, err := InstallTrackedRepo(source, sourcePath, trackOpts)
if err != nil {
if !opts.Quiet {
ui.StepFail(displayName, err.Error())
}
return trackedInstallOutcome{failed: true}
}
if opts.DryRun {
return trackedInstallOutcome{action: trackedResult.Action}
}
if !opts.Quiet {
ui.StepDone(displayName, fmt.Sprintf("installed (tracked, %d skills)", trackedResult.SkillCount))
}
skills := trackedResult.Skills
if len(skills) == 0 {
skills = []string{displayName}
}
return trackedInstallOutcome{skills: skills, skillCount: trackedResult.SkillCount}
}
// installPlainFromConfig installs a single non-tracked skill from config.
// Returns true on success, false on failure (UI messages emitted internally).
func installPlainFromConfig(
ctx InstallContext,
source *Source,
sourcePath string,
destPath string,
displayName string,
bareName string,
groupDir string,
opts InstallOptions,
) bool {
if err := validate.SkillName(bareName); err != nil {
if !opts.Quiet {
ui.StepFail(displayName, fmt.Sprintf("invalid name: %v", err))
}
return false
}
// Ensure group directory exists.
if groupDir != "" {
if err := sourcefs.MkdirAllIn(sourcePath, filepath.FromSlash(groupDir), opts.SourceFollow); err != nil {
if !opts.Quiet {
ui.StepFail(displayName, fmt.Sprintf("failed to create group directory: %v", err))
}
return false
}
}
result, err := Install(source, destPath, opts)
if err != nil {
if !opts.Quiet {
ui.StepFail(displayName, err.Error())
}
return false
}
if opts.DryRun {
if !opts.Quiet {
ui.StepDone(displayName, result.Action)
}
return true
}
if err := ctx.PostInstallSkill(displayName); err != nil {
ui.Warning("post-install hook failed for %s: %v", displayName, err)
}
if !opts.Quiet {
ui.StepDone(displayName, "installed")
}
return true
}