Every API request reloads the skills and agents metadata stores and
swaps the server's pointers under s.mu, but several handlers read or
wrote the stores outside that lock: listing, skill detail, check, the
update stream, and the hash refresh after setting targets. Concurrent
requests could race on the pointer or on the stores' maps, which the
race detector reported hundreds of times under load and which can
crash the server with a concurrent map access.
Store access now always happens under s.mu. Reads return a copy of
the entry so it can be used after unlocking, and target handlers hash
files outside the lock and only write the result under it.
InstallOptions.Force carried two meanings: "overwrite what is already
there" and "proceed despite audit findings". Update paths set Force
unconditionally to overwrite a staged temp dir, so they had to leave it
off to keep the audit gate alive -- which swallowed the caller's --force
and made the gate impossible to override from either the CLI or the web
UI.
Split the audit intent into InstallOptions.AuditOverride and route every
gate through it alone. Entry points translate --force explicitly, so a
missed one degrades to "force has no effect" rather than "audit is
bypassed". Scan failures stay fail-closed regardless: accepting known
findings is not the same as being unable to learn any.
This also closes two paths that bypassed the gate outright, since both
inherited Force for overwrite reasons alone: install --json and grouped
batch updates.
In the web UI, Force Retry now appears only on failures force can
actually resolve -- a permission-denied unlink previously offered a
button that could never succeed -- and CLI-only flag hints are stripped
from messages rendered there.
Behaviour change: a --force at install time no longer exempts later
updates from the audit gate; it is a one-off decision per command.
When tracked repo metadata is committed but the gitignored clone dirs are
absent on a fresh machine, `update --all` silently reported zero work: the
batch path skipped missing repos with no message, and the web UI's
update-all paths only listed repos present on disk.
Missing tracked repos are now detected and surfaced everywhere:
- CLI: a warning block (human output) plus an aggregated
`missing_tracked_repos` summary (names + one-shot rehydrate hint) in JSON;
the per-item error is concise. Fixed for both global and project modes,
and batch and single-target paths.
- Server: GET /api/update/missing-tracked-repos and the update-all/stream
summaries report them; POST /api/update/rehydrate re-clones them from
metadata (mirrors bare `skillshare install`).
- UI: the Update page shows a warning banner listing missing repos with a
Rehydrate button; strings added to all 11 locales.
Closes#212
Make the v0.19.16 sources map actually do something: route all path
reads through Config.EffectiveSkillsSource() / EffectiveAgentsSource() /
EffectiveExtrasSource() so users who set sources.skills / sources.agents /
sources.extras get the configured paths everywhere.
Mechanical changes (~50 files):
- cmd/skillshare/**: replace cfg.Source / cfg.ExtrasSource reads with
cfg.EffectiveSkillsSource() / cfg.EffectiveExtrasSource()
- internal/server/**: replace s.cfg.Source / s.cfg.ExtrasSource reads
with the same helpers; server.skillsSource() / gitignoreDir() use the
global-mode helper
- internal/config/config_validate.go: validate using EffectiveSkillsSource
while still rejecting configs that set neither field explicitly
- internal/config/reconcile.go, registry.go: read via helpers
Behavioral fixes (paired with the migration):
- extras init / extras source / POST /api/extras no longer silently
backfill the legacy extras_source field — the runtime fallback inside
EffectiveExtrasSource() handles it. This stops surprise diffs in users'
config.yaml after running extras commands.
- extras source command writes back to whichever field the user already
configured (sources.extras if set, otherwise legacy extras_source) so
the update is not shadowed.
- source_type in /api/extras and extras list correctly reports "default"
vs "extras_source" by inspecting the raw (unresolved) configured value;
sources.extras counts as configured for this purpose.
UI:
- ui/src/lib/fieldDocs.ts: neutralize sources / sources.* descriptions
so they describe both project and global modes (Phase 5).
Preserved (intentional):
- Direct writes to cfg.Source / cfg.AgentsSource / cfg.ExtrasSource in
init, migrate.go, cloneForSave, and server.reloadConfig — these are
raw-field writes, not path reads.
- Updated test expectations for the removed backfills.
Addresses P1 and P2 from code review on the RWMutex commit:
P1 — Stream handlers now snapshot config under RLock before streaming:
- handleDiffStream: clone Targets map to prevent concurrent map panic
- handleCheckStream, handleAuditStream: snapshot source/projectRoot
- handleUpdateStream: snapshot source (per-item Lock already correct)
P2 — All read handlers now use narrow snapshot instead of defer RUnlock:
- RLock → copy needed fields to locals → RUnlock → do work with locals
- Prevents slow I/O (git fetch, version check, audit scan) from blocking
the auto-reload middleware's exclusive Lock on concurrent requests
Other changes:
- computeTargetDiff takes source parameter (no more s.cfg.Source access)
- Add cloneTargets() helper for safe map copying
- Add writeOpsLogTo/writeAuditLogTo for lock-free oplog writes
Replace sequential POST loops and blocking useQuery calls with
Server-Sent Events for real-time progress feedback on Update, Check,
and Diff operations. Extract shared helpers to reduce duplication:
- initSSE() in middleware.go consolidates SSE preamble across 4 handlers
- createSSEStream() in client.ts unifies EventSource lifecycle management
- StreamProgressBar component replaces 3 near-identical progress bars
- handleDiff now delegates to computeTargetDiff (eliminates ~150 lines)
- Paginate audit results (20 cards + 10 findings per card) to prevent
DOM overload with large skill counts