mirror of
https://github.com/rohitg00/skillkit.git
synced 2026-11-18 18:04:26 +00:00
feat/eval-system
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7e3b31b43b |
feat: add session intelligence — timeline, handoff, and lineage (#65)
* feat: add session intelligence — timeline, handoff, and lineage commands Three new skill-centric session intelligence features: - `skillkit timeline` — unified chronological event stream merging skill executions, git commits, observations, decisions, and snapshots - `skillkit session handoff` — generates structured markdown for agent-to-agent context transfer with accomplished/pending tasks, key files, observations, and recommendations - `skillkit lineage` — tracks skill impact graph connecting skills to files, commits, and observations with aggregate stats Includes 44 tests, updated docs/website to reflect 44 agents, and CodeRabbit review fixes (defensive access, type validation, double-count prevention). * fix: address CodeRabbit review round 2 - Fix primer.mdx example: "29 more agents" → "41 more agents" - Validate invalid since date in timeline (return empty instead of silent filter-all) - Guard hist.durationMs with Number.isFinite before formatting - Fix misleading "commits" label → "changes" for activity log entries |
||
|
|
aafa086314 |
feat: add skillkit doctor command and enhance skillkit status (#64)
* feat: add `skillkit doctor` command and enhance `skillkit status` - New `skillkit doctor` command with 10 health checks across 3 categories (environment, configuration, skills) with --json and --fix flags - Enhanced `skillkit status` shows project overview dashboard when no active session exists instead of bare "No active session found" message - Overview includes agent, config, version, skills count, recent history * feat: add skill activity log, session snapshots, and session explain Three new session intelligence features: - Activity Log: tracks which skills were active per git commit (`skillkit activity` with --skill, --limit, --json flags) - Session Snapshots: save/restore named snapshots of session state (`skillkit session snapshot save|restore|list|delete`) - Session Explain: structured summary of current session (`skillkit session explain` with --json, --no-git flags) Includes path traversal protection on snapshot names, YAML validation on parsed activity data, and robust error handling on snapshot restore. 983 tests passing (942 core + 41 CLI including 14 E2E tests). * feat: add issue planner - generate plans from GitHub Issues New `skillkit issue plan` and `skillkit issue list` commands that fetch GitHub Issues via `gh` CLI and generate StructuredPlans. Parses checkboxes, file mentions, and labels from issue body. 29 new tests, all passing. * fix: address CodeRabbit review suggestions for PR #64 - doctor.ts: use execFileSync, typed AgentType, remove as-any casts - activity.ts: validate parseInt with radix and NaN guard - issue.ts: wrap JSON.parse in try/catch for gh CLI output - issue-planner.ts: guard JSON.parse, fix includeTests no-op ternary, remove duplicate IssuePlanMetadata (import from types.ts) - status.ts: typed StatusOverview interface, remove as-any casts - snapshot-manager.ts: align observations param with SessionSnapshot type - session.ts: proper type mapping for observations (no double cast) - activity-log.ts: min-length guard for SHA prefix match - e2e tests: use top-level imports instead of inline require() - commands.test.ts: add assertions for DoctorCommand, IssuePlanCommand, IssueListCommand * fix: address review feedback from CodeRabbit and Devin - Include test step in no-checklist fallback when includeTests is true - Restore observations from snapshot in SessionSnapshotRestoreCommand * fix: address remaining CodeRabbit and Devin review feedback - doctor.ts: Use getAdapter() per AgentType instead of getAllAdapters() spread (which destroys prototype methods like isDetected) - doctor.ts: Use getGlobalSkillsDir() from AGENT_CONFIG instead of hardcoded path formula - session-explainer.ts: Defensively default state.history and state.decisions to empty arrays - session-explainer.ts: Use loadConfig().agent with skillSource fallback instead of always using skillSource as agent name - observation-store.ts: Add static readAll() that reads observations without sessionId mismatch clearing - session.ts: Use ObservationStore.readAll() for snapshot save to avoid empty observations due to sessionId mismatch |
||
|
|
4257e8555b |
feat: Memory, Mesh Network with Security, Messaging, MCP Server (#27)
* feat: add persistent memory, mesh network, and agent messaging (v2.0.0)
Add three major features to SkillKit:
@skillkit/memory - CozoDB-backed semantic memory with embeddings
- 384-dim embeddings via @xenova/transformers (all-MiniLM-L6-v2)
- Memory tiers (warm/long) with promotion logic based on access patterns
- HNSW semantic search for context retrieval
- Storage: ~/.skillkit/agents/{agentId}/memory.db
@skillkit/mesh - Peer mesh network for multi-machine agent distribution
- Local LAN discovery via UDP multicast (239.255.255.250)
- Tailscale VPN support for secure remote connections
- Bidirectional peer registration with health checks
- HTTP and WebSocket transports
- Storage: ~/.skillkit/hosts.json
@skillkit/messaging - Inter-agent messaging system
- Inbox/sent/archived storage with JSON persistence
- Message router for local and cross-host delivery
- Priority levels (low/normal/high/urgent)
- Message types (request/response/notification/update)
- Storage: ~/.skillkit/messages/{inbox|sent|archived}/{agentId}/
CLI Commands:
- skillkit mesh: init, add, remove, list, health, discover, status
- skillkit message: send, inbox, read, reply, archive, forward, sent, status
All packages bumped to version 2.0.0
* feat: add advanced learning, automation, and workflow features
Implements comprehensive enhancement plan with 9 phases:
Phase 1: @skillkit/resources Package
- Bundled agents (9): architect, build-error-resolver, code-reviewer,
doc-updater, e2e-runner, planner, refactor-cleaner, security-reviewer, tdd-guide
- Command templates (10): /tdd, /plan, /e2e, /learn, /build-fix, etc.
- Operational profiles (4): dev, review, research, security
- Guidelines (5): security, code-style, testing, git, performance
- Hook templates (12): typescript-check, eslint-check, security-scan, etc.
Phase 2-3: Pattern Learning & Confidence Scoring
- skillkit learn: Extract patterns from git history
- skillkit pattern status|feedback|approve|reject|export|import|cluster
- Confidence evolution: success +0.05, failure -0.10
Phase 4: Session State Persistence
- skillkit session status|start|load|list|note|complete|in-progress
- Cross-session context preservation in ~/.skillkit/sessions/
Phase 5: Workflow Pipelines
- skillkit workflow pipeline list|<name>
- Built-in: feature, bugfix, refactor, security-audit, documentation
Phase 6: Operational Profiles
- skillkit profile list|<name>|create|remove
- Mode switching for different work contexts
Phase 7: Guidelines System
- skillkit guideline list|show|enable|disable|create|remove
- Always-on coding standards
Phase 8: Hook Templates
- skillkit hook template list|apply|show
- Pre-built automation patterns
Phase 9: Slash Command Templates
- skillkit command available|install <id>
- Install bundled command templates
* feat: add MCP memory server for Claude/Cursor integration
Adds @skillkit/mcp-memory package - an MCP (Model Context Protocol) server
that exposes SkillKit's persistent memory system to Claude, Cursor, and
other MCP-compatible AI tools.
Tools:
- memory_store: Store memories with semantic embeddings
- memory_search: Semantic search through memories
- memory_recall: Recall by category/tier/tags
- memory_get: Get memory by ID
- memory_forget: Delete a memory
- memory_link: Create relationships between memories
- memory_reinforce: Strengthen/weaken memories
- memory_stats: Get memory statistics
Resources:
- memory://stats: Current statistics (JSON)
- memory://recent: Recent memories (JSON)
Usage with Claude Desktop:
```json
{
"mcpServers": {
"skillkit-memory": {
"command": "npx",
"args": ["@skillkit/mcp-memory"],
"env": { "SKILLKIT_AGENT_ID": "claude-desktop" }
}
}
}
```
* feat(mesh): add comprehensive security hardening
- Add cryptographic identity layer (Ed25519 keypairs, fingerprints)
- Add message encryption (XChaCha20-Poly1305) and signing
- Add secure keystore with AES-256-GCM at-rest encryption
- Add JWT authentication and challenge-response handshake
- Add secure WebSocket (WSS) and HTTPS transports
- Add signed discovery messages with trust modes (open/signed/trusted-only)
- Add TLS certificate management (self-signed)
- Add CLI commands: mesh security init/status, mesh peer trust/revoke/list
- Add security presets: development, signed, secure, strict
- Align all package versions to 1.7.11
Security stack: @noble/ed25519, @noble/curves, @noble/ciphers, @noble/hashes, jose
* docs(website): add mesh network and security features
- Add Mesh Network feature card with P2P description
- Add Security feature card with Ed25519/TLS details
- Add Mesh command group with security commands
* fix: address CodeRabbit review suggestions
- cli/mesh: validate port input (1-65535 range check)
- memory/cozo-adapter: call db.close() before nullifying reference
- memory/schema: fix primary key definitions (id => columns)
- resources/hooks: guard pre-commit hooks against empty file lists
- mcp-memory: prevent double-initialization with memoized promise
- mcp-memory: don't mask reinforce failures as "not found"
- mcp-memory: guard main() to prevent auto-start on import
- mcp-memory: add js language identifiers to README code blocks
- mesh/test: update MESH_VERSION test expectation to 1.7.11
* fix: address additional CodeRabbit and Devin review suggestions
Memory package:
- encoder: reset initPromise on failure to allow retry
- encoder: call pipeline.dispose() in disposeEncoder to free GPU resources
- schema: fix probe query to use named-column matching for correct arity
Mesh package:
- hosts-config: add file locking to prevent race conditions
- local discovery: reduce multicast TTL to 1 for LAN-only traffic
- local discovery: handle announce errors in setInterval
- local discovery: close socket on membership setup failure
- local discovery: validate inbound packets (type, hostId, port range)
- local discovery: use rinfo.address to prevent spoofing
- health: guard against overlapping health checks with checking flag
- websocket: prevent auto-reconnect on explicit disconnect()
* fix(memory): increase semantic search fetch multiplier for better filter coverage
Increase from 2x to 5x results to reduce likelihood of incomplete
results when agentId/category/tier filters exclude many candidates.
Filters are applied in application code after HNSW vector search.
* feat(cli): add 'add' as alias for 'install' command
* fix(mesh/auth): use asymmetric EdDSA signing for JWT tokens
Tokens are now signed with the sender's Ed25519 private key and
verified using the sender's public key embedded in the token.
This fixes the issue where tokens couldn't be verified across
different AuthManager instances/hosts because each had a random
symmetric secret.
Changes:
- Switch from HS256 (symmetric) to EdDSA (asymmetric) algorithm
- Include sender's publicKey in JWT payload
- Add privateKey/privateKeyHex getters to PeerIdentity
- Verify fingerprint matches publicKey before accepting token
|
||
|
|
d3ffb79512 |
feat: add session memory system for cross-agent learning (#3)
* feat: add session memory system for cross-agent learning
Implements a lightweight, local-first memory system that works across
all 17 AI coding agents. Key features:
- YAML-based storage (no databases or external services)
- Three memory layers: Session, Project, and Global
- AI-powered compression of observations into learnings
- Cross-agent memory injection with relevance matching
- Progressive disclosure for token efficiency
Core modules:
- ObservationStore: Capture raw session observations
- LearningStore: Persist compressed learnings
- MemoryCompressor: AI/rule-based compression engine
- MemoryInjector: Context-aware memory retrieval
- MemoryObserver: Automatic observation capture
CLI commands (skillkit memory):
- status, search, list, show, compress, export
- import, clear, add, rate, config
TUI integration:
- New Memory screen with tabs (Learnings/Observations/Search)
- useMemory hook for state management
- Updated navigation and keybindings
Code quality improvements:
- Replaced nested ternaries with explicit functions
- Added named constants for magic numbers
- Simplified navigation with lookup map
* feat: add marketplace, testing, CI/CD, workflows, and fix dry-run bug
New Features:
- Skill Marketplace: Aggregates skills from GitHub repos with caching
- Skill Testing Framework: 18+ assertion types for skill validation
- CI/CD Integration: Templates for GitHub Actions, GitLab CI, CircleCI
- Workflow Orchestration: Wave-based parallel skill execution
- Session Management: Pause/resume support with state persistence
- Agent Execution: CLI spawning for claude/codex/gemini/opencode/goose
New CLI Commands:
- skillkit marketplace - Browse and install from marketplace
- skillkit test - Run skill tests with assertions
- skillkit run - Execute skills with orchestration
- skillkit status - Show session state
- skillkit pause/resume - Session management
- skillkit workflow - Workflow management
New TUI Screens:
- Marketplace browser with search
- History viewer with session details
- Workflow management screen
Bug Fix:
- Fixed dry-run mode in memory compress command (Devin review)
The --dry-run flag now properly prevents storing learnings
* fix: add missing yaml dependency to @skillkit/cli
* chore: update pnpm-lock.yaml
* fix: address 7 bugs identified by Devin code review
- Fix MemoryIndexStore ignoring isGlobal flag (memory-index.ts)
- Fix ESM runtime crash using require('yaml') in test runner (runner.ts)
- Fix ESM runtime crash using require('node:fs') in MarketplaceAggregator (aggregator.ts)
- Fix ESM runtime crash using require('node:fs') in SessionManager (manager.ts)
- Fix memory rate command parsing rating from --content instead of positional arg (memory.ts)
- Fix SkillExecutionEngine emitting failed status for paused executions (engine.ts)
- Fix run command misclassifying directories as skill markdown files (run.ts)
* fix: address CodeRabbit/Devin review issues (batch 2)
- Fix resume skipping checkpoint handling in engine.ts
- Add paused status guard before completeExecution call
- Fix NaN% progress when totalSteps=0 in Execute.tsx
- Fix effectiveness ?? instead of || to preserve 0% values in compressor.ts
- Clear cache on source change in aggregator.ts addSource/removeSource
* fix: address critical and major CodeRabbit/Devin issues (batch 3)
Security fixes:
- Remove shell: true from spawn to prevent shell injection (agents.ts)
- Add safe regex test with ReDoS protection (engine.ts)
- Add security warning for verification commands (engine.ts)
Bug fixes:
- Fix global memory index path to use ~/.skillkit/ (compressor.ts, memory-index.ts)
- Add task_complete events and verification to resumeExecution (engine.ts)
- Handle absolute URLs in skill entries (aggregator.ts)
- Implement actual port availability check (runner.ts)
- Remove invalid JUnit report format (templates.ts)
- Add crash guard to workflow list command (workflow/list.ts)
* fix: address remaining CodeRabbit/Devin issues (batch 4)
- ReDoS protection: Add pattern validator to reject dangerous regex patterns
(nested quantifiers, overlapping alternations, excessive quantifiers)
- Add missing 'cancelled' status message in execution engine
- Make completedAt/durationMs optional in TaskExecutionResult
- Fix tag filter to skip untagged tests when tags are specified
- Improve error-solution matching: require keyword similarity + solution indicator
- Validate --limit and sanitize --tags in memory/marketplace CLI commands
- Guard against missing arrays when importing YAML learnings
- Escape YAML frontmatter values to prevent invalid output
- Preserve original learning source when merging learnings
- Fix keyword double-counting in memory index search
- Normalize keywords by stripping punctuation for better matching
|