Files
skillshare/internal/server/handler_diff_stream.go
T
Willie c638b46096 fix(sync): replace copy-mode copies with links when switching back to merge (#506)
* fix(sync): replace copy-mode copies with links when switching back to merge

Switching a target from copy back to merge left every copy whose entry name
did not change as a real folder. Merge sync treated it as the user's own
folder, so status and doctor kept saying "run skillshare sync" while sync
changed nothing, and only `sync --force` turned it into a link.

Copy mode already replaces merge-mode links with copies on its own, so merge
sync now replaces a folder the manifest records with a checksum (a copy-mode
copy) with a link, without --force. Folders the manifest does not record, or
records as a link, stay untouched. The pre-sync target backup is unchanged.

The CLI and dashboard diffs name these entries as copies sync replaces
instead of suggesting --force, and the sync-modes docs describe the switch.

* fix(sync): keep copy-mode copies the user edited when switching to merge

The first version replaced any folder the manifest recorded as a copy-mode
copy. The manifest checksum is only a record, so a copy edited since copy
mode made it was deleted too, and `sync --json` and project-mode dashboard
sync take no pre-sync backup to restore it from.

Replace the folder only while it still hashes to the recorded checksum
(Manifest.OwnsCopy). An edited copy is kept and `sync --force` still
replaces it; diff keeps suggesting --force for it. A copy whose name: was
rewritten by prefixed naming, or that was made with custom ignore patterns,
no longer hashes to the source checksum and is kept the same way.

* fix(ui): say a copy-mode copy in a merge target is replaced by a link

The Sync page picks a row's text from the diff item's action alone. The
copy-mode copy that merge sync now replaces comes from the server as an
update, which the page worded as "Link is broken or points elsewhere. Will
be relinked" even though no link exists yet.

Match the server's reason and show "Copy made by copy mode. Will be replaced
by a link" instead, in all eleven dashboard languages.

* docs(sync): say only unedited copy-mode copies are replaced on merge sync

The sync-modes paragraph promised that copies copy mode made are replaced
with links without --force. Merge sync keeps a copy that no longer hashes to
the checksum copy mode recorded, so an edited copy, or one whose name: copy
mode rewrote for prefixed naming, still needs --force.

State the three cases in sync-modes and qualify the diff Modified and Local
override labels, in English and the ja, ko, zh-Hans and zh-Hant copies.

* fix(sync): keep the copy until its replacement link exists

Replacing a copy-mode copy removed the folder first and then created the
link. If creating the link failed (no symlink or junction rights on a
Windows or network filesystem), sync returned an error with the skill gone
from the target. This path now also runs on a plain sync, not only on
--force.

Park the folder beside the entry, create the link, then delete the parked
folder; on failure the folder is moved back. A parked folder left by a run
that died mid-way may be the only copy, so the next sync stops with a message
instead of deleting it. The manifest also forgets the copy-mode mtime and
naming records of an entry that is now a link.

* test(sourcefs): register replaceWithLink's target writes in the raw-write ratchet

The previous commit moved the merge-mode folder replacement into
replaceWithLink, which the ratchet reports as new raw os calls and the old
os.RemoveAll allowance as stale. All of them act on the sync target, not the
skills source, like the neighbouring entries.

* fix(sync): bound the parked copy's name and document the naming exception

The folder parked while a link is created was named after the entry, so an
entry near the 255-byte filename limit (a long flattened nested skill)
could not be moved aside and its replacement failed. The parked name is now
a fixed-length hash of the entry name.

sync-modes also says what happens when `target_naming` changes in the same
sync: the copy gets a new entry name and the prune pass removes it as an
orphan whether or not it was edited, as it did before this change.
2026-10-10 00:13:29 +08:00

253 lines
9.9 KiB
Go

package server
import (
"maps"
"net/http"
"os"
"path/filepath"
"skillshare/internal/config"
ssync "skillshare/internal/sync"
"skillshare/internal/utils"
)
// handleDiffStream serves an SSE endpoint that streams diff computation progress.
// Events:
// - "discovering" → {"phase":"..."} immediately on connect
// - "start" → {"total": N} after discovery (N = target count)
// - "result" → diffTarget per-target diff result
// - "done" → {"diffs":[...]} final payload (same shape as GET /api/diff)
func (s *Server) handleDiffStream(w http.ResponseWriter, r *http.Request) {
safeSend, ok := initSSE(w)
if !ok {
return
}
ctx := r.Context()
// Snapshot config under RLock, then release before slow I/O.
s.mu.RLock()
source := s.cfg.EffectiveSkillsSource()
agentsSource := s.agentsSource()
globalMode := s.cfg.Mode
ignorePatterns := ssync.EffectiveFileIgnorePatterns(s.cfg.Ignore)
targets := s.cloneTargets()
s.mu.RUnlock()
if globalMode == "" {
globalMode = "merge"
}
safeSend("discovering", map[string]string{"phase": "scanning source directory"})
discovered, ignoreStats, err := ssync.DiscoverSourceSkillsWithStats(source, s.skillsWalk())
if err != nil {
safeSend("error", map[string]string{"error": err.Error()})
return
}
safeSend("start", map[string]int{"total": len(targets)})
var diffs []diffTarget
checked := 0
for name, target := range targets {
select {
case <-ctx.Done():
return
default:
}
dt := s.computeTargetDiff(name, target, discovered, globalMode, source, ignorePatterns)
diffs = append(diffs, dt)
checked++
safeSend("result", map[string]any{
"diff": dt,
"checked": checked,
})
}
diffs = s.appendAgentDiffs(diffs, targets, agentsSource, "")
donePayload := map[string]any{"diffs": diffs}
maps.Copy(donePayload, ignorePayload(ignoreStats))
maps.Copy(donePayload, agentIgnorePayload(agentsSource, nil))
safeSend("done", donePayload)
}
// computeTargetDiff computes the diff for a single target.
// Extracted from handleDiff to share logic with the stream handler.
func (s *Server) computeTargetDiff(name string, target config.TargetConfig, discovered []ssync.DiscoveredSkill, globalMode, source string, ignorePatterns []string) diffTarget {
sc := target.SkillsConfig()
mode := sc.Mode
if mode == "" {
mode = globalMode
}
dt := diffTarget{Target: name, Items: make([]diffItem, 0)}
if !sc.IsEnabled() {
return dt // skills off: nothing is synced, so nothing differs
}
if mode == "symlink" {
status := ssync.CheckStatus(sc.Path, source)
if status != ssync.StatusLinked {
dt.Items = append(dt.Items, diffItem{Skill: "(entire directory)", Action: "link", Reason: "source only", Kind: kindSkill})
}
return dt
}
filtered, err := ssync.SelectTargetSkills(discovered, name, sc)
if err != nil {
return dt
}
resolution, err := ssync.ResolveTargetSkillsForTarget(name, config.ResourceTargetConfig{
Path: sc.Path,
TargetNaming: sc.TargetNaming,
}, filtered)
if err != nil {
dt.Items = append(dt.Items, diffItem{Skill: "(target naming)", Action: "skip", Reason: err.Error(), Kind: kindSkill})
return dt
}
// Surface collision/validation stats so the UI can show why skills were skipped
dt.CollisionCount = len(resolution.Collisions)
dt.SkippedCount = len(filtered) - len(resolution.Skills)
validNames := resolution.ValidTargetNames()
manifest, _ := ssync.ReadManifest(sc.Path)
legacyNames := resolution.LegacyNames(mode, sc.Path, manifest)
renamedFrom := ssync.RenamedFrom(legacyNames)
if mode == "copy" {
for _, resolved := range resolution.Skills {
skill := resolved.Skill
oldChecksum, isManaged := manifest.Managed[resolved.TargetName]
targetSkillPath := filepath.Join(sc.Path, resolved.TargetName)
if !isManaged {
if info, statErr := os.Stat(targetSkillPath); statErr == nil {
if old, ok := renamedFrom[resolved.TargetName]; ok {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "kept", Reason: ssync.KeptLegacyReason(old), Kind: kindSkill})
} else if info.IsDir() {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "skip", Reason: "local copy (sync --force to replace)", Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "target entry is not a directory", Kind: kindSkill})
}
} else if old, ok := renamedFrom[resolved.TargetName]; ok && os.IsNotExist(statErr) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.RenameReason(old), Kind: kindSkill})
} else if os.IsNotExist(statErr) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "link", Reason: "source only", Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "cannot access target entry", Kind: kindSkill})
}
} else {
targetInfo, statErr := os.Stat(targetSkillPath)
if os.IsNotExist(statErr) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "link", Reason: "missing (deleted from target)", Kind: kindSkill})
} else if statErr != nil {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "cannot access target entry", Kind: kindSkill})
} else if !targetInfo.IsDir() {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "target entry is not a directory", Kind: kindSkill})
} else if recorded := manifest.Naming[resolved.TargetName]; recorded != "" && recorded != resolution.Naming {
// A copy made under another naming carries another name:, so sync re-copies it.
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.NamingChangedReason, Kind: kindSkill})
} else {
oldMtime := manifest.Mtimes[resolved.TargetName]
currentMtime, mtimeErr := ssync.DirMaxMtimeWithIgnore(skill.SourcePath, ignorePatterns)
if mtimeErr == nil && oldMtime > 0 && currentMtime == oldMtime {
continue
}
srcChecksum, checksumErr := ssync.DirChecksumWithIgnore(skill.SourcePath, ignorePatterns)
if checksumErr != nil {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "cannot compute checksum", Kind: kindSkill})
} else if srcChecksum != oldChecksum {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "content changed", Kind: kindSkill})
}
}
}
}
for managedName := range manifest.Managed {
if _, keepLegacy := legacyNames[managedName]; keepLegacy {
continue
}
if !validNames[managedName] {
dt.Items = append(dt.Items, diffItem{Skill: managedName, Action: "prune", Reason: "orphan copy", Kind: kindSkill})
}
}
return dt
}
// Merge mode
for _, resolved := range resolution.Skills {
skill := resolved.Skill
targetSkillPath := filepath.Join(sc.Path, resolved.TargetName)
_, err := os.Lstat(targetSkillPath)
if err != nil {
if old, ok := renamedFrom[resolved.TargetName]; ok && os.IsNotExist(err) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.RenameReason(old), Kind: kindSkill})
} else if os.IsNotExist(err) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "link", Reason: "source only", Kind: kindSkill})
}
continue
}
if utils.IsSymlinkOrJunction(targetSkillPath) {
absLink, linkErr := utils.ResolveLinkTarget(targetSkillPath)
if linkErr != nil {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "link target unreadable", Kind: kindSkill})
continue
}
absSource, _ := filepath.Abs(skill.SourcePath)
if !utils.PathsEqual(absLink, absSource) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: "symlink points elsewhere", Kind: kindSkill})
}
} else if old, ok := renamedFrom[resolved.TargetName]; ok {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "kept", Reason: ssync.KeptLegacyReason(old), Kind: kindSkill})
} else if manifest.OwnsCopy(resolved.TargetName, targetSkillPath) {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "update", Reason: ssync.CopyToLinkReason, Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: resolved.TargetName, Action: "skip", Reason: "local copy (sync --force to replace)", Kind: kindSkill})
}
}
// Orphan check
entries, _ := os.ReadDir(sc.Path)
for _, entry := range entries {
eName := entry.Name()
if manifest.SkipsHidden(eName) {
continue
}
if _, keepLegacy := legacyNames[eName]; keepLegacy {
continue
}
entryPath := filepath.Join(sc.Path, eName)
if !validNames[eName] {
info, statErr := os.Lstat(entryPath)
if statErr != nil {
continue
}
if utils.IsSymlinkOrJunction(entryPath) {
absLink, linkErr := utils.ResolveLinkTarget(entryPath)
if linkErr != nil {
continue
}
absSource, _ := filepath.Abs(source)
if utils.PathHasPrefix(absLink, absSource+string(filepath.Separator)) {
dt.Items = append(dt.Items, diffItem{Skill: eName, Action: "prune", Reason: "orphan symlink", Kind: kindSkill})
}
} else if info.IsDir() {
if _, inManifest := manifest.Managed[eName]; inManifest {
dt.Items = append(dt.Items, diffItem{Skill: eName, Action: "prune", Reason: "orphan managed directory (manifest)", Kind: kindSkill})
} else {
if resolution.Naming == "flat" && (utils.HasNestedSeparator(eName) || utils.IsTrackedRepoDir(eName)) {
dt.Items = append(dt.Items, diffItem{Skill: eName, Action: "prune", Reason: "orphan managed directory", Kind: kindSkill})
} else {
dt.Items = append(dt.Items, diffItem{Skill: eName, Action: "local", Reason: "local only", Kind: kindSkill})
}
}
}
}
}
return dt
}