Files
skillshare/internal/server/handler_diff_stream.go
T
1419b62499 fix(sync): never delete or re-point entries skillshare did not create (#914)
* fix(sync): never delete or re-point entries skillshare did not create

On Windows, a plain `init` followed by `sync` deleted things in
~/.claude/skills that skillshare never created, with no backup and no
prompt. It also re-pointed another tool's junction. Seen in a sandbox
run of v0.30.7:

- prune removed real folders whose names only look like skillshare's
  flat naming (team__shared, _drafts), via the name heuristic
- prune removed a broken link that another tool or the user made
- merge sync deleted and re-created a same-name link that pointed into
  another manager's store
- init's import deleted original folders that matched the source
  byte for byte

The manifest already records what skillshare made, so use it:

- prune keeps a look-alike folder that is not in the manifest and warns
  once per sync; --force still removes it
- prune removes a broken external link only if the manifest records it
  (e.g. after the source moved); otherwise it keeps the link and warns
- merge sync re-points a link only if the manifest records it or it
  already points into the source; otherwise it reports the entry as
  kept local, as it already does for a real folder; --force replaces it
- init's import moves identical copies to the trash under
  replaced-by-init/<target>/ instead of deleting them

Tests: ownership_test.go adds regression tests that use createLink, so
they run on Windows without Developer Mode (junctions). The 4 keep
tests fail without this change. TestSyncTargetMerge_FixesBrokenLink now
records the link in the manifest first, since a same-name link that
skillshare did not make is no longer re-pointed. Existing tests in
internal/sync, internal/trash and cmd/skillshare show the same failures
on a Windows host with and without the change (symlink privilege). The
devcontainer suite has not been run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: say what sync and init keep when skillshare did not create it

The previous commit makes sync and init leave alone entries skillshare
did not create. Update the pages that describe that behaviour:

- sync-modes: merge mode keeps a same-name link from another tool and
  reports it as local; orphan cleanup keeps folders that only look like
  skillshare's flat names, and broken links outside the source that the
  manifest does not record. sync --force still removes them.
- init: identical folders replaced with links during the first sync go
  to the trash first (replaced-by-init/<tool>/, cleaned after 7 days).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(sourcefs): drop the raw-write allowance replaceIdenticalCopies no longer needs

replaceIdenticalCopies now moves an identical copy to the trash instead
of calling os.RemoveAll, so TestRawWriteRatchet reports its allowance as
stale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(sync): apply the ownership rule at every consumer

Review of the first commit found places that still did, or predicted,
the old behaviour. One rule now decides whether a link is skillshare's:
Manifest.OwnsLink (the manifest records a link there, or it points into
the source). It compares canonical paths, so a source reached through an
alias and a relative project link still count as skillshare's.

- copy mode kept re-pointing: it removed any same-name link and wrote a
  copy. It now keeps a link it does not own, like merge mode.
- the dashboard diff predicted a prune for a folder that is only named
  like skillshare's, and an update for another tool's link. It now
  reports them as local and kept. The orphan-link prediction uses the
  same rule as prune.
- `diff` reported a target as in sync while sync printed "kept local"
  for another tool's link. It lists the link as a local override.
- doctor reported a broken link skillshare did not create as an error
  and suggested `skillshare sync`, which now keeps it. It is a warning
  with its own row, and `broken_symlinks` is `warning` in --json.
- `trash restore` put a folder init had replaced into the source as a
  new nested skill. Restore refuses entries under replaced-by-init/ and
  names the trash folder to copy from.
- init reported a folder it could not move to the trash as one that
  differs from the source, and dropped the error. It is reported
  separately, with the error.
- the PruneOptions.ManagedOnly comment still said broken external links
  are skipped; manifest-recorded ones are removed.

Tests cover the states the review listed: a source behind an alias, a
relative project link, ManagedOnly, standard naming, dry run with
--force, and a recorded copy the user replaced with a link.
TestSync_Pruning_RemovesBrokenExternalSymlinks passed without testing
anything, because FileExists follows links; it is now two tests that
use Lstat, one per owner.

Agent merge is unchanged: agents have no link manifest.

* fix(diff): list another tool's link as kept in copy mode too

Copy mode now keeps a same-name link skillshare did not create, but both
diffs read the entry with os.Stat. A broken link showed as a skill to
add on every run, and a live one as a plain local copy, while sync kept
both and said "kept local".

`diff` and the dashboard diff check the link first and report it the way
merge mode does: a local override, kept until --force.

* test(sync): make the aliased-source link while its folder exists

TestSyncMergeRepointsUnrecordedLinkIntoAliasedSource failed on Windows:
it linked straight to a missing folder, and a junction cannot be made
to one. It now links first and removes the folder after, like the other
broken-link tests in this file.

* fix(sync): compare a broken link by where it would point

linksIntoSource resolved both paths with EvalSymlinks, which gives up on
a path that does not exist. A broken link is exactly that, so a link
into a source reached through an alias (a symlinked config folder, or a
short 8.3 path on Windows) compared an unresolved path with a resolved
one and was taken for another tool's link. Sync then kept it instead of
repairing or pruning it.

It now uses sourcewalk.Canonical, which resolves the nearest folder that
exists and keeps the missing tail. The test covers both spellings: the
alias in the link, and the alias in the configured source.

* fix(init): report what the first sync kept, and match prune in the diff

Two previews said less, or more, than sync does.

init ran its first sync quietly and printed only "Synced". Sync now keeps
a link another tool put on a skill's name, a folder that is only named
like skillshare's, and a broken link it did not create, so the skill was
reported as synced while the tool still loaded the other copy. firstSync
returns the warnings a plain sync prints, and init shows them. They
already name the folders that differ from the source, so the separate
list of those is gone.

The dashboard diff counted an orphan link as a removal whenever the
manifest recorded it. Prune keeps a recorded link that still resolves
outside the source (the source moved and the old checkout stayed), so
the Sync page showed a removal that never happened. The diff now uses
prune's own rule: a link into the source goes when broken or recorded;
any other link only when broken and recorded.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Willie <egg358358@gmail.com>
2026-11-17 04:15:04 +08:00

258 lines
11 KiB
Go

package server
import (
"maps"
"net/http"
"os"
"path/filepath"
"skillshare/internal/config"
ssync "skillshare/internal/sync"
"skillshare/internal/utils"
)
// handleDiffStream serves an SSE endpoint that streams diff computation progress.
// Events:
// - "discovering" → {"phase":"..."} immediately on connect
// - "start" → {"total": N} after discovery (N = target count)
// - "result" → diffTarget per-target diff result
// - "done" → {"diffs":[...]} final payload (same shape as GET /api/diff)
func (s *Server) handleDiffStream(w http.ResponseWriter, r *http.Request) {
safeSend, ok := initSSE(w)
if !ok {
return
}
ctx := r.Context()
// Snapshot config under RLock, then release before slow I/O.
s.mu.RLock()
source := s.cfg.EffectiveSkillsSource()
agentsSource := s.agentsSource()
globalMode := s.cfg.Mode
ignorePatterns := ssync.EffectiveFileIgnorePatterns(s.cfg.Ignore)
targets := s.cloneTargets()
s.mu.RUnlock()
if globalMode == "" {
globalMode = "merge"
}
safeSend("discovering", map[string]string{"phase": "scanning source directory"})
discovered, ignoreStats, err := ssync.DiscoverSourceSkillsWithStats(source, s.skillsWalk())
if err != nil {
safeSend("error", map[string]string{"error": err.Error()})
return
}
safeSend("start", map[string]int{"total": len(targets)})
var diffs []diffTarget
checked := 0
for name, target := range targets {
select {
case <-ctx.Done():
return
default:
}
dt := s.computeTargetDiff(name, target, discovered, globalMode, source, ignorePatterns)
diffs = append(diffs, dt)
checked++
safeSend("result", map[string]any{
"diff": dt,
"checked": checked,
})
}
diffs = s.appendAgentDiffs(diffs, targets, agentsSource, "")
donePayload := map[string]any{"diffs": diffs}
maps.Copy(donePayload, ignorePayload(ignoreStats))
maps.Copy(donePayload, agentIgnorePayload(agentsSource, nil))
safeSend("done", donePayload)
}
// computeTargetDiff computes the diff for a single target.
// Extracted from handleDiff to share logic with the stream handler.
func (s *Server) computeTargetDiff(name string, target config.TargetConfig, discovered []ssync.DiscoveredSkill, globalMode, source string, ignorePatterns []string) diffTarget {
sc := target.SkillsConfig()
mode := sc.Mode
if mode == "" {
mode = globalMode
}
dt := diffTarget{Target: name, Items: make([]diffItem, 0)}
if !sc.IsEnabled() {
return dt // skills off: nothing is synced, so nothing differs
}
if mode == "symlink" {
status := ssync.CheckStatus(sc.Path, source)
if status != ssync.StatusLinked {
dt.Items = append(dt.Items, diffItem{Skill: "(entire directory)", Action: "link", Reason: "source only", Kind: kindSkill})
}
return dt
}
resolution, err := ssync.ResolveTargetSkillsForTarget(name, sc, discovered)
if err != nil {
dt.Items = append(dt.Items, diffItem{Skill: "(target naming)", Action: "skip", Reason: err.Error(), Kind: kindSkill})
return dt
}
// Surface collision/validation stats so the UI can show why skills were skipped
dt.CollisionCount = len(resolution.Collisions)
dt.SkippedCount = resolution.SkippedCount
dt.NamingSkippedCount = resolution.NamingSkippedCount
dt.Warnings = resolution.Warnings
for _, warning := range resolution.SelectionWarnings {
dt.Items = append(dt.Items, diffItem{Skill: "(skill selection)", Action: "skip", Reason: warning, Kind: kindSkill})
}
validNames := resolution.ValidTargetNames()
manifest, _ := ssync.ReadManifest(sc.Path)
legacyNames := resolution.LegacyNames(mode, sc.Path, manifest)
renamedFrom := ssync.RenamedFrom(legacyNames)
if mode == "copy" {
for _, resolved := range resolution.Skills {
skill := resolved.Skill
oldChecksum, isManaged := manifest.Managed[resolved.TargetName]
targetSkillPath := filepath.Join(sc.Path, resolved.TargetName)
// Another tool's link on the skill's name: sync keeps it, like a local copy.
if utils.IsSymlinkOrJunction(targetSkillPath) && !manifest.OwnsLink(resolved.TargetName, targetSkillPath, source) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "skip", Reason: ssync.ForeignLinkReason, Kind: kindSkill})
continue
}
if !isManaged {
if info, statErr := os.Stat(targetSkillPath); statErr == nil {
if old, ok := renamedFrom[resolved.TargetName]; ok {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "kept", Reason: ssync.KeptLegacyReason(old), Kind: kindSkill})
} else if info.IsDir() {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "skip", Reason: "local copy (sync --force to replace)", Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "target entry is not a directory", Kind: kindSkill})
}
} else if old, ok := renamedFrom[resolved.TargetName]; ok && os.IsNotExist(statErr) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.RenameReason(old), Kind: kindSkill})
} else if os.IsNotExist(statErr) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "link", Reason: "source only", Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "cannot access target entry", Kind: kindSkill})
}
} else {
targetInfo, statErr := os.Stat(targetSkillPath)
if os.IsNotExist(statErr) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "link", Reason: "missing (deleted from target)", Kind: kindSkill})
} else if statErr != nil {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "cannot access target entry", Kind: kindSkill})
} else if !targetInfo.IsDir() {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "target entry is not a directory", Kind: kindSkill})
} else if recorded := manifest.Naming[resolved.TargetName]; recorded != "" && recorded != resolution.Naming {
// A copy made under another naming carries another name:, so sync re-copies it.
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.NamingChangedReason, Kind: kindSkill})
} else {
oldMtime := manifest.Mtimes[resolved.TargetName]
currentMtime, mtimeErr := ssync.DirMaxMtimeWithIgnore(skill.SourcePath, ignorePatterns)
if mtimeErr == nil && oldMtime > 0 && currentMtime == oldMtime {
continue
}
srcChecksum, checksumErr := ssync.DirChecksumWithIgnore(skill.SourcePath, ignorePatterns)
if checksumErr != nil {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "cannot compute checksum", Kind: kindSkill})
} else if srcChecksum != oldChecksum {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "content changed", Kind: kindSkill})
}
}
}
}
for managedName := range manifest.Managed {
if _, keepLegacy := legacyNames[managedName]; keepLegacy {
continue
}
if !validNames[managedName] {
dt.Items = append(dt.Items, diffItem{Skill: managedName, Action: "prune", Reason: "orphan copy", Kind: kindSkill})
}
}
return dt
}
// Merge mode
for _, resolved := range resolution.Skills {
skill := resolved.Skill
targetSkillPath := filepath.Join(sc.Path, resolved.TargetName)
_, err := os.Lstat(targetSkillPath)
if err != nil {
if old, ok := renamedFrom[resolved.TargetName]; ok && os.IsNotExist(err) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.RenameReason(old), Kind: kindSkill})
} else if os.IsNotExist(err) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "link", Reason: "source only", Kind: kindSkill})
}
continue
}
if utils.IsSymlinkOrJunction(targetSkillPath) {
absLink, linkErr := utils.ResolveLinkTarget(targetSkillPath)
if linkErr != nil {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "link target unreadable", Kind: kindSkill})
continue
}
absSource, _ := filepath.Abs(skill.SourcePath)
if !utils.PathsEqual(absLink, absSource) {
if manifest.OwnsLink(resolved.TargetName, targetSkillPath, source) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "symlink points elsewhere", Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "skip", Reason: ssync.ForeignLinkReason, Kind: kindSkill})
}
}
} else if old, ok := renamedFrom[resolved.TargetName]; ok {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "kept", Reason: ssync.KeptLegacyReason(old), Kind: kindSkill})
} else if manifest.OwnsCopy(resolved.TargetName, targetSkillPath) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.CopyToLinkReason, Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "skip", Reason: "local copy (sync --force to replace)", Kind: kindSkill})
}
}
// Orphan check
entries, _ := os.ReadDir(sc.Path)
for _, entry := range entries {
eName := entry.Name()
if manifest.SkipsHidden(eName) {
continue
}
if _, keepLegacy := legacyNames[eName]; keepLegacy {
continue
}
entryPath := filepath.Join(sc.Path, eName)
if !validNames[eName] {
info, statErr := os.Lstat(entryPath)
if statErr != nil {
continue
}
if utils.IsSymlinkOrJunction(entryPath) {
// As prune does: a link into the source goes once it is broken or
// while the manifest records it; a link to anywhere else only when
// it is both broken and recorded.
_, statErr := os.Stat(entryPath)
broken := os.IsNotExist(statErr)
_, inManifest := manifest.Managed[eName]
if inSource := ssync.LinksIntoSource(entryPath, source); (inSource && (broken || inManifest)) || (!inSource && broken && inManifest) {
dt.Items = append(dt.Items, diffItem{Skill: eName, Action: "prune", Reason: "orphan symlink", Kind: kindSkill})
}
} else if info.IsDir() {
if _, inManifest := manifest.Managed[eName]; inManifest {
dt.Items = append(dt.Items, diffItem{Skill: eName, Action: "prune", Reason: "orphan managed directory (manifest)", Kind: kindSkill})
} else {
// Sync keeps a folder the manifest does not record, whatever its name.
dt.Items = append(dt.Items, diffItem{Skill: eName, Action: "local", Reason: "local only", Kind: kindSkill})
}
}
}
}
return dt
}