Compare commits

..
12 Commits
Author SHA1 Message Date
Rohit Ghumare 549a21ad84 chore: release v1.10.0 2026-02-02 23:33:12 +00:00
Rohit Ghumare 109f00cd95 feat(core): add QMD-inspired hybrid search pipeline (#40)
* feat(core): add QMD-inspired hybrid search pipeline

Implement a hybrid search system combining vector embeddings with keyword search:

- Vector embeddings using node-llama-cpp with GGUF models (nomic-embed-text)
- RRF (Reciprocal Rank Fusion) for combining multiple rankers
- Query expansion with LLM and synonym fallback
- Position-aware score blending (different weights for top3, top10, rest)
- SQLite storage with sqlite-vec extension (falls back to in-memory)
- Auto-download of local GGUF models (~2GB)

New search module:
- embeddings.ts: Vector embedding generation service
- vector-store.ts: SQLite-based vector storage
- rrf.ts: RRF fusion algorithm
- expansion.ts: Query expansion with caching
- hybrid.ts: Main hybrid search pipeline
- local-models.ts: GGUF model management

CLI integration:
- skillkit recommend --hybrid: Enable hybrid search
- skillkit recommend --expand: Enable query expansion
- skillkit recommend --rerank: Enable LLM re-ranking
- skillkit recommend --build-index: Build embedding index

Tests: 56 new tests for search module (all passing)

* fix(search): address CodeRabbit review findings

- Add missing beforeEach import in embeddings.test.ts
- Update optional dependencies to current versions
- Fix weights/variations length mismatch in expansion.ts
- Avoid mutating input arrays in rrf.ts mergeRankings
- Delete orphaned chunk vectors in vector-store.ts delete() and clear()
- Add comment for empty catch block in vector-store.ts
- Make hybridSearch helper use buildIndex for true hybrid search
- Handle graceful degradation when embeddings unavailable
- Add node-llama-cpp mock in hybrid.test.ts

* fix(search): address CodeRabbit review round 2

- Fix gemma GGUF URL to point to valid community repo (bartowski)
- Add SQL injection protection for table names in vector-store.ts
- Fix resource leak with try/finally in hybridSearch helper
- Add backpressure handling for large model downloads
- Clamp startLine to non-negative in chunk calculation
- Add runtime validation warning for --expand/--rerank without --hybrid
- Update node-llama-cpp API usage (createEmbeddingContext, createContext)
- Update pnpm-lock.yaml for new optional dependency versions

* fix(search): use @ts-ignore for optional node-llama-cpp imports

CI doesn't install optional dependencies, so @ts-ignore is needed
instead of @ts-expect-error to suppress module not found errors.
2026-02-02 23:28:08 +00:00
Rohit Ghumare e3d7e0e28f feat(marketplace): add beautiful-mermaid skill
Add beautiful-mermaid to the skills marketplace:
- Render Mermaid diagrams as beautiful SVGs or ASCII art
- 5 diagram types: flowchart, sequence, class, ER, state
- 15 built-in themes + Shiki/VS Code theme integration
- Zero DOM dependencies, works in Node.js/Bun/Deno/browsers

Source: rohitg00/beautiful-mermaid
2026-02-02 17:42:09 +00:00
Rohit Ghumare 6cbbdabf59 feat(website): add dynamic stats from npm and GitHub APIs (#39)
* feat(website): add dynamic stats from npm and GitHub APIs

- Create useStats hook to fetch real-time version, downloads, and stars
- Replace hardcoded values in App.tsx stats bar with dynamic data
- Replace hardcoded version in Hero.tsx badge with dynamic data
- Add 5-minute cache to reduce API calls
- Fallback to default values if APIs fail

* fix: address CodeRabbit and Devin review comments

- Fix duplicate API calls: remove useStats from Hero, pass version as prop from App
- Fix falsy checks: use typeof checks for downloads and stargazers_count
  to correctly handle zero values instead of showing defaults
2026-02-02 17:38:48 +00:00
Rohit Ghumare c1c179217d fix(website): update version to 1.9.0 2026-02-02 17:09:24 +00:00
Rohit Ghumare de4425f440 chore: release v1.9.0 (#38)
Features:
- Hierarchical skill tree with 12 categories (skillkit tree)
- LLM-based reasoning engine for skill discovery
- Explainable recommendations (--explain, --reasoning flags)
- Connector placeholders for tool-agnostic skills (~~CRM, ~~chat, etc.)
- Execution flow tracking with metrics
- Standalone vs Enhanced mode detection

CLI:
- New `tree` command for taxonomy navigation
- Enhanced `recommend` with explanation support
- Fixed --skill alias and agent detection

Documentation:
- Added tree.mdx for Skill Tree documentation
- Updated recommendations.mdx with reasoning features
- Updated TUI docs with tree view mode
- Updated README with new features
- Updated website Features and Commands
2026-02-02 16:48:43 +00:00
Rohit Ghumare 8121c2bbac feat(core): add PageIndex-inspired patterns for skill discovery (#37)
* feat(core): add PageIndex-inspired patterns for skill discovery

Implements patterns from anthropics/knowledge-work-plugins:

Tree Module (packages/core/src/tree/):
- Hierarchical skill taxonomy with 12 categories
- Tree generator from skill tags and metadata
- Tree serializer for JSON/Markdown export
- Related skills graph with 4 relation types

Reasoning Engine (packages/core/src/reasoning/):
- LLM-based tree traversal for skill discovery
- Explainable recommendations with reasoning chains
- Search planning with category relevance scoring
- Result caching with 5-minute TTL

Connectors (packages/core/src/connectors/):
- Tool-agnostic ~~ placeholder system
- 13 connector categories (crm, chat, email, etc.)
- Auto-suggest mappings from MCP config
- Placeholder detection and replacement utilities

Execution Flow (packages/core/src/execution/):
- Step-by-step execution tracking with metrics
- Automatic retry with exponential backoff
- Standalone vs Enhanced mode detection
- Capability-based feature gating

CLI Integration:
- New `skillkit tree` command with --generate, --stats flags
- Enhanced `skillkit recommend` with --explain, --reasoning flags

TUI Integration:
- Tree view mode in Marketplace (toggle with 'v')
- Arrow key navigation for tree hierarchy

* fix: address CodeRabbit and Devin review feedback

- Use os.homedir() instead of process.env.HOME || '~' fallback
  (tree.ts, tree.service.ts, mode.ts)
- Add guard for division by zero in tree stats percentage
- Validate parseInt result for --depth flag
- Fix average stats calculation to use cacheMisses count
- Extract category from data in validateCategoryScore
- Remove unused skillMap field from TreeGenerator
- Make reasoning provider configurable instead of hardcoded
- Add try/catch for tree loading errors in Marketplace
- Handle missing tree nodes during traversal

* fix(execution): clear timeout timer to prevent resource leak

Store setTimeout ID and clear it in finally block when step
completes successfully before timeout. This prevents orphaned
timers from accumulating during long-running applications.
2026-02-02 16:18:13 +00:00
Rohit Ghumare 3565daa7f1 fix(cli): add --skill alias and fix agent detection
- Add --skill as alias for --skills option (install command)
- Fix sync command to always detect agent instead of using default 'universal'
- Auto-install npm dependencies for skills with package.json
2026-02-02 12:07:41 +00:00
Rohit Ghumare 8496e3c63c feat(publish): add RFC 8615 well-known skills support (#36)
* feat(agents): add skill-to-subagent converter command

Add `skillkit agent from-skill` command to convert SkillKit skills into
Claude Code native subagent format (.md files in .claude/agents/).

Features:
- Reference mode (default): generates subagent with `skills: [skill-name]`
- Inline mode (--inline): embeds full skill content in system prompt
- Options: --model, --permission, --global, --output, --dry-run

New files:
- packages/core/src/agents/skill-converter.ts
- packages/core/src/agents/__tests__/skill-converter.test.ts (23 tests)

Closes #22

* feat(publish): add RFC 8615 well-known skills support

Redesign publish workflow to align with industry standard:

- Add WellKnownProvider for auto-discovery from any domain
- skillkit add https://example.com discovers skills via /.well-known/skills/
- skillkit publish generates well-known hosting structure
- skillkit publish submit opens GitHub issue (legacy workflow)

Provider discovers skills from:
- /.well-known/skills/index.json (manifest)
- /.well-known/skills/{skill-name}/SKILL.md (skill files)

Includes 16 tests for WellKnownProvider and structure generation.

* fix(wellknown): correct baseSkillsUrl calculation for skills.json fallback

When using the /.well-known/skills.json fallback URL, the previous
logic produced a malformed URL with duplicated .well-known path segment:
- Before: https://example.com/.well-known/.well-known/skills
- After: https://example.com/.well-known/skills

Extract calculateBaseSkillsUrl helper function for better testability
and add 4 new unit tests covering both URL formats.

* docs: update publish command documentation for well-known skills

- README.md: Add self-hosting section with well-known URI structure
- commands.mdx: Add Publishing Commands section
- marketplace.mdx: Expand publish section with self-hosting instructions
- skills.mdx: Add self-hosting workflow documentation

Documents the new `skillkit publish` workflow that generates RFC 8615
well-known URI structures for decentralized skill hosting.

* fix(security): address path traversal and CRLF vulnerabilities

- agent.ts: Sanitize skill.name when building default filename
- publish.ts: Add sanitizeSkillName() to validate skill names before
  using in file paths, verify resolved paths stay within target dir
- skill-converter.ts: Normalize CRLF to LF before extracting body
  content to handle Windows-style line endings
- wellknown.ts: Add sanitizeSkillName() to validate remote skill names,
  verify resolved paths stay within temp dir, encode URL components

* fix(agents): improve YAML string escaping logic

Only escape strings that actually need it for valid YAML parsing:
- Strings containing newlines or colons
- Strings starting with special YAML characters (-, *, &, !, {, [, >, |, @, `)
- Strings starting with quotes

This fixes unnecessary escaping of strings like "code-quality" where
hyphens in the middle don't need escaping.

* fix(wellknown): address security review feedback

- Use path.sep instead of hardcoded '/' for cross-platform compatibility
- Add sanitization and path containment check in generateWellKnownStructure
- Remove unused 'vi' import in wellknown.test.ts
2026-02-01 21:38:42 +00:00
Rohit Ghumare 847af0f7ab feat(agents): add skill-to-subagent converter command (#34)
* feat(agents): add skill-to-subagent converter command

Add `skillkit agent from-skill` command to convert SkillKit skills into
Claude Code native subagent format (.md files in .claude/agents/).

Features:
- Reference mode (default): generates subagent with `skills: [skill-name]`
- Inline mode (--inline): embeds full skill content in system prompt
- Options: --model, --permission, --global, --output, --dry-run

New files:
- packages/core/src/agents/skill-converter.ts
- packages/core/src/agents/__tests__/skill-converter.test.ts (23 tests)

Closes #22

* fix(agents): address code review feedback

- Update CLI option descriptions to list all valid values (model: inherit, permission: bypassPermissions)
- Add filename sanitization to prevent path traversal attacks via --output
- Fix parseAllowedTools to handle both string and YAML array formats
- Support space-separated tool lists in allowed-tools
- Add tests for array and space-separated formats
2026-02-01 20:00:11 +00:00
Rohit Ghumare 34e8cd0b61 docs(tui): update documentation for Solid.js migration (#33)
Update TUI documentation to reflect the v1.8.0 migration from React to
Solid.js:

- Change @opentui/react to @opentui/solid
- Change React 19.0.0 to Solid.js 1.9.0
- Add note about migration rationale (performance, bundle size)
- Update performance optimizations section with Solid.js primitives
  (createMemo, createSignal, createEffect)
- Add windowed list rendering mention
- Update Integration section with Solid.js code example
- Update graceful exit to reference onCleanup instead of React unmount
2026-02-01 17:55:45 +00:00
Rohit Ghumare cbd9fcd679 chore: fix version mismatch in root and docs packages 2026-02-01 17:34:40 +00:00
80 changed files with 10937 additions and 297 deletions
+77 -5
View File
@@ -142,8 +142,25 @@ Filter by task, category, or minimum score:
skillkit recommend --search "auth" # Task-based search
skillkit recommend --category security # Category filter
skillkit recommend --min-score 80 # Quality threshold
skillkit recommend --explain # Show WHY skills match
skillkit recommend --reasoning # Use LLM-based search
```
### Hierarchical Skill Tree
Browse skills organized in a navigable taxonomy:
```bash
skillkit tree # Show full tree
skillkit tree Frontend # Show Frontend subtree
skillkit tree "Frontend > React" # Navigate to subcategory
skillkit tree --generate # Generate tree from index
skillkit tree --stats # Show tree statistics
skillkit tree --depth 2 # Limit display depth
```
**Categories:** Development, Frontend, Backend, Mobile, DevOps, Testing, Security, AI/ML, Database, Tooling, Documentation, Performance
### Session Memory System
Your AI agents learn, but that knowledge dies with the session. SkillKit captures learnings and makes them persistent:
@@ -190,7 +207,7 @@ skillkit ui
skillkit
```
**Navigation:** `h` Home | `b` Browse | `r` Recommend | `t` Translate | `c` Context | `l` List | `s` Sync | `q` Quit
**Navigation:** `h` Home | `m` Marketplace | `b` Browse | `v` Tree View | `r` Recommend | `t` Translate | `c` Context | `l` List | `s` Sync | `q` Quit
### Skill Testing Framework
@@ -298,6 +315,10 @@ skillkit status # Show skill and agent status
```bash
skillkit recommend # Get smart recommendations
skillkit recommend --explain # Show reasoning for matches
skillkit recommend --reasoning # Use LLM-based discovery
skillkit tree # Browse hierarchical taxonomy
skillkit tree --generate # Generate tree from index
skillkit marketplace # Browse skill marketplace
skillkit marketplace search # Search marketplace
skillkit find <query> # Quick skill search
@@ -428,11 +449,31 @@ skillkit methodology load # Load a methodology
### Publishing & Sharing
```bash
skillkit publish # Publish skill to marketplace
skillkit publish # Generate well-known hosting structure
skillkit publish submit # Submit to SkillKit marketplace
skillkit create # Create new skill
skillkit init # Initialize in project
```
#### Self-Hosting Skills (RFC 8615)
Generate a well-known URI structure to host skills on your own domain:
```bash
# Generate hosting structure
skillkit publish ./my-skills --output ./public
# Users install from your domain
skillkit add https://your-domain.com
```
This creates:
```
.well-known/skills/
index.json # Skill manifest
my-skill/SKILL.md # Skill files
```
### Configuration
```bash
@@ -491,8 +532,29 @@ import {
// Recommendations
RecommendationEngine,
ReasoningRecommendationEngine,
analyzeProject,
// Tree & Taxonomy
TreeGenerator,
generateSkillTree,
loadTree,
buildSkillGraph,
getRelatedSkills,
// Reasoning Engine
ReasoningEngine,
createReasoningEngine,
// Connectors (Tool-Agnostic Placeholders)
detectPlaceholders,
replacePlaceholders,
suggestMappingsFromMcp,
// Execution Flow
ExecutionManager,
detectExecutionMode,
// Context
ContextManager,
syncToAllAgents,
@@ -516,10 +578,20 @@ import {
const skill = await translateSkill(skillContent, 'cursor');
console.log(skill.content);
// Example: Get recommendations
const engine = new RecommendationEngine();
// Example: Get recommendations with reasoning
const engine = new ReasoningRecommendationEngine();
await engine.initReasoning();
const profile = await analyzeProject('./my-project');
const recs = engine.recommend(profile);
const recs = await engine.recommendWithReasoning(profile, { reasoning: true });
// Example: Browse skill tree
const tree = generateSkillTree(skills);
console.log(tree.rootNode.children); // Categories
// Example: Detect execution mode
const mode = detectExecutionMode();
console.log(mode.mode); // 'standalone' or 'enhanced'
console.log(mode.capabilities); // Available MCP capabilities
```
## Skill Sources & Attribution
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "skillkit",
"version": "1.8.1",
"version": "1.10.0",
"description": "Supercharge AI coding agents with portable skills. Install, translate, and share skills across Claude Code, Cursor, Codex, Copilot & 13 more",
"type": "module",
"bin": {
+7
View File
@@ -48,10 +48,12 @@ import {
AICommand,
AuditCommand,
PublishCommand,
PublishSubmitCommand,
AgentCommand,
AgentListCommand,
AgentShowCommand,
AgentCreateCommand,
AgentFromSkillCommand,
AgentTranslateCommand,
AgentSyncCommand,
AgentValidateCommand,
@@ -95,6 +97,7 @@ import {
GuidelineDisableCommand,
GuidelineCreateCommand,
GuidelineRemoveCommand,
TreeCommand,
} from '@skillkit/cli';
const __filename = fileURLToPath(import.meta.url);
@@ -149,10 +152,12 @@ cli.register(CommandCmd);
cli.register(AICommand);
cli.register(AuditCommand);
cli.register(PublishCommand);
cli.register(PublishSubmitCommand);
cli.register(AgentCommand);
cli.register(AgentListCommand);
cli.register(AgentShowCommand);
cli.register(AgentCreateCommand);
cli.register(AgentFromSkillCommand);
cli.register(AgentTranslateCommand);
cli.register(AgentSyncCommand);
cli.register(AgentValidateCommand);
@@ -213,4 +218,6 @@ cli.register(GuidelineDisableCommand);
cli.register(GuidelineCreateCommand);
cli.register(GuidelineRemoveCommand);
cli.register(TreeCommand);
cli.runExit(process.argv.slice(2));
+9 -1
View File
@@ -233,10 +233,18 @@ skillkit init # Initialize project
skillkit create <name> # Create new skill
skillkit validate [path] # Validate skill format
skillkit read <skills> # Read skill content
skillkit publish # Publish to marketplace
skillkit settings --set key=value # Configure settings
```
## Publishing Commands
```bash
skillkit publish [path] # Generate well-known hosting structure
skillkit publish --output dir # Output to specific directory
skillkit publish --dry-run # Preview without writing
skillkit publish submit # Submit to SkillKit marketplace
```
## Interactive TUI
```bash
+32 -1
View File
@@ -48,8 +48,39 @@ skillkit install anthropics/skills --agent claude-code,cursor
## Publish Skills
### Self-Host on Your Domain (Recommended)
Generate a well-known URI structure (RFC 8615) to host skills on your own domain:
```bash
# Create and validate your skill
skillkit create my-skill
skillkit validate my-skill
skillkit publish
# Generate hosting structure
skillkit publish ./my-skill --output ./public
```
This creates:
```
.well-known/skills/
index.json # Skill manifest for auto-discovery
my-skill/
SKILL.md # Your skill content
```
Deploy the `.well-known` folder to your web server. Users can then install via:
```bash
skillkit add https://your-domain.com
```
### Submit to SkillKit Marketplace
To submit your skill for inclusion in the central marketplace:
```bash
skillkit publish submit
```
This opens a GitHub issue for review by maintainers.
+1
View File
@@ -9,6 +9,7 @@
"agents",
"translation",
"marketplace",
"tree",
"recommendations",
"primer",
"memory",
+41 -1
View File
@@ -32,6 +32,33 @@ skillkit recommend --category security
skillkit recommend --min-score 80
```
## Explainable Recommendations
See WHY skills match your project:
```bash
skillkit recommend --explain
```
Output:
```
vercel-react-best-practices (Score: 92)
├── Matched: React, TypeScript, Next.js
├── Your stack: Next.js 14, React 18
└── Path: Frontend > React > Best Practices
```
## LLM-Based Reasoning
Use reasoning-based discovery for complex queries:
```bash
skillkit recommend --reasoning
```
This uses an LLM to traverse the skill taxonomy and find the most relevant skills based on semantic understanding.
## How It Works
The engine analyzes:
@@ -50,9 +77,22 @@ Scores based on:
## Programmatic API
```typescript
import { RecommendationEngine, analyzeProject } from '@skillkit/core'
import {
RecommendationEngine,
ReasoningRecommendationEngine,
analyzeProject
} from '@skillkit/core'
// Basic recommendations
const engine = new RecommendationEngine()
const profile = await analyzeProject('./my-project')
const recs = engine.recommend(profile)
// With reasoning and explanations
const reasoningEngine = new ReasoningRecommendationEngine()
await reasoningEngine.initReasoning()
const explained = await reasoningEngine.recommendWithReasoning(profile, {
reasoning: true,
explain: true
})
```
+18 -2
View File
@@ -94,6 +94,22 @@ Or manually create a `SKILL.md` file following the format above.
# Test locally
skillkit validate ./my-skill
# Publish to marketplace
skillkit publish
# Generate well-known hosting structure for self-hosting
skillkit publish ./my-skill --output ./public
# Or submit to SkillKit marketplace
skillkit publish submit
```
### Self-Hosting
The `skillkit publish` command generates an RFC 8615 well-known URI structure:
```
.well-known/skills/
index.json # Manifest for auto-discovery
my-skill/
SKILL.md # Skill content
```
Deploy to your domain and users can install via `skillkit add https://your-domain.com`.
+144
View File
@@ -0,0 +1,144 @@
---
title: Skill Tree
description: Browse skills in a hierarchical taxonomy
---
# Skill Tree
Browse and navigate skills organized in a hierarchical taxonomy structure.
## Quick Start
```bash
# Show full skill tree
skillkit tree
# Navigate to a category
skillkit tree Frontend
# Navigate to subcategory
skillkit tree "Frontend > React"
```
## Commands
### Display Tree
```bash
skillkit tree # Full tree
skillkit tree Development # Specific category
skillkit tree --depth 2 # Limit display depth
```
### Generate Tree
```bash
skillkit tree --generate # Generate from skill index
```
This builds a hierarchical tree from the skill index based on tags and metadata.
### Export Formats
```bash
skillkit tree --json # JSON output
skillkit tree --markdown # Markdown output
```
### Statistics
```bash
skillkit tree --stats
```
Output:
```
Overview:
Total Skills: 15,064
Categories: 156
Max Depth: 4
Top-Level Categories:
Development 4,521 skills (30.0%)
Frontend (1,245), Backend (1,102), Mobile (456)
DevOps 2,340 skills (15.5%)
CI/CD (890), Kubernetes (567), Docker (445)
AI/ML 1,890 skills (12.5%)
LLM Integration (756), Agents (445), RAG (312)
```
## Categories
The tree organizes skills into 12 top-level categories:
| Category | Description |
|----------|-------------|
| Development | General coding, programming patterns |
| Frontend | React, Vue, Angular, Svelte, UI/UX |
| Backend | Node.js, Python, Go, APIs, servers |
| Mobile | React Native, Flutter, iOS, Android |
| DevOps | CI/CD, Kubernetes, Docker, cloud |
| Testing | Unit, E2E, integration, TDD |
| Security | Auth, encryption, vulnerabilities |
| AI/ML | LLM, agents, RAG, embeddings |
| Database | SQL, NoSQL, ORM, migrations |
| Tooling | Linting, formatting, bundling |
| Documentation | API docs, guides, READMEs |
| Performance | Optimization, caching, profiling |
## TUI Tree View
In the TUI, press `v` to toggle tree view in the Marketplace:
- **Arrow keys** - Navigate tree
- **Enter** - Expand category / select skill
- **←** - Go back / collapse
- **v** - Toggle list/tree view
## Related Skills Graph
Find skills related to each other:
```typescript
import { buildSkillGraph, getRelatedSkills } from '@skillkit/core'
const graph = buildSkillGraph(skills)
const related = getRelatedSkills('react-patterns', graph, skills, {
limit: 5,
types: ['similar', 'complementary']
})
```
### Relation Types
- **similar** - Shares tags and functionality
- **complementary** - Works well together
- **dependency** - Required by other skills
- **alternative** - Different approach to same problem
## Programmatic API
```typescript
import {
TreeGenerator,
generateSkillTree,
loadTree,
saveTree,
treeToMarkdown
} from '@skillkit/core'
// Generate tree from skills
const generator = new TreeGenerator({ maxDepth: 3 })
const tree = generator.generateTree(skills)
// Or use the helper function
const tree = generateSkillTree(skills)
// Save and load
saveTree(tree, './skill-tree.json')
const loaded = loadTree('./skill-tree.json')
// Export to markdown
const markdown = treeToMarkdown(tree)
```
+41 -9
View File
@@ -18,10 +18,12 @@ skillkit
## Framework
Built on modern technologies:
- **@opentui/react** - Terminal rendering
- **React 19.0.0** - Component architecture
- **@opentui/solid** - Terminal rendering with Solid.js
- **Solid.js 1.9.0** - Reactive component architecture
- **Bun 1.2.0+** - Runtime requirement
The TUI was migrated from React to Solid.js in v1.8.0 for improved performance and smaller bundle size.
## Screens
| Screen | Key | Purpose |
@@ -103,6 +105,20 @@ Browse 15,000+ skills:
- Featured skills
- 3-phase animation loading
- Quality scoring
- **Tree View** - Press `v` to toggle hierarchical navigation
#### Tree View Mode
Navigate skills in a hierarchical taxonomy:
| Key | Action |
|-----|--------|
| `v` | Toggle tree/list view |
| `→` or `Enter` | Expand category |
| `←` | Collapse / go back |
| `↑`/`↓` | Navigate |
Categories include: Development, Frontend, Backend, Mobile, DevOps, Testing, Security, AI/ML, Database, Tooling, Documentation, Performance
### Recommend Screen
@@ -159,8 +175,10 @@ loadSkills()
### Performance Optimizations
- `useMemo` for filtered lists
- `useCallback` for event handlers
- `createMemo` for filtered lists and computed values
- `createSignal` for reactive state management
- `createEffect` for side effects and subscriptions
- Windowed list rendering for large datasets
- Pagination limits rendered items
- Progressive animation with phases
@@ -174,18 +192,32 @@ loadSkills()
## Integration
The TUI integrates with @skillkit/core:
The TUI integrates with @skillkit/core using Solid.js reactive patterns:
```typescript
findAllSkills(searchDirs) // Returns skill metadata
evaluateSkillDirectory(path) // Quality metrics
getQualityGrade(score) // 0-100 to A-F
import { createSignal, createMemo, createEffect } from 'solid-js';
const [skills, setSkills] = createSignal([]);
createEffect(async () => {
const result = await findAllSkills(searchDirs);
setSkills(result);
});
const filteredSkills = createMemo(() =>
skills().filter(s => s.name.includes(searchTerm()))
);
```
Core functions:
- `findAllSkills(searchDirs)` - Returns skill metadata
- `evaluateSkillDirectory(path)` - Quality metrics
- `getQualityGrade(score)` - 0-100 to A-F
## Graceful Exit
The TUI handles shutdown gracefully:
1. Unmounts React components
1. Runs Solid.js cleanup functions (`onCleanup`)
2. Restores terminal state
3. Cleans up signal handlers
4. Exits process cleanly
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "skillkit-docs",
"version": "1.8.0",
"version": "1.10.0",
"private": true,
"scripts": {
"build": "next build",
+7 -4
View File
@@ -12,6 +12,7 @@ import { Attribution } from './components/Attribution';
import { AdvancedFeatures } from './components/AdvancedFeatures';
import { UseCases } from './components/UseCases';
import { TeamEnterprise } from './components/TeamEnterprise';
import { useStats } from './hooks/useStats';
const GITHUB_ICON = (
<svg className="w-5 h-5" fill="currentColor" viewBox="0 0 24 24">
@@ -34,6 +35,8 @@ function scrollToSection(e: React.MouseEvent, sectionId: string): void {
}
export default function App(): React.ReactElement {
const stats = useStats();
return (
<div className="min-h-screen text-zinc-100 font-sans selection:bg-white selection:text-black" style={{ backgroundColor: '#000000' }}>
<nav className="fixed top-0 left-0 right-0 z-50 border-b border-zinc-800 backdrop-blur-md" style={{ backgroundColor: 'rgba(0,0,0,0.9)' }}>
@@ -102,7 +105,7 @@ export default function App(): React.ReactElement {
</nav>
<main className="pt-14">
<Hero />
<Hero version={stats.version} />
<div className="border-b border-zinc-800/50 py-2.5" style={{ background: 'linear-gradient(to bottom, rgba(9,9,11,0.95), rgba(0,0,0,1))' }}>
<div className="mx-auto max-w-5xl px-4 sm:px-6 lg:px-8">
@@ -114,7 +117,7 @@ export default function App(): React.ReactElement {
className="flex items-center gap-1.5 text-zinc-500 hover:text-white transition-colors group"
>
<span className="text-zinc-600 group-hover:text-zinc-400">v</span>
<span className="text-white font-medium">1.8.0</span>
<span className="text-white font-medium">{stats.version}</span>
</a>
<span className="text-zinc-800">·</span>
<a
@@ -126,7 +129,7 @@ export default function App(): React.ReactElement {
<svg className="w-3 h-3 text-zinc-600 group-hover:text-zinc-400" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 16v1a3 3 0 003 3h10a3 3 0 003-3v-1m-4-4l-4 4m0 0l-4-4m4 4V4" />
</svg>
<span className="text-white font-medium">2.4k</span>
<span className="text-white font-medium">{stats.downloads}</span>
</a>
<span className="text-zinc-800">·</span>
<a
@@ -138,7 +141,7 @@ export default function App(): React.ReactElement {
<svg className="w-3 h-3 text-zinc-600 group-hover:text-yellow-500" fill="currentColor" viewBox="0 0 24 24">
<path d="M12 2l3.09 6.26L22 9.27l-5 4.87 1.18 6.88L12 17.77l-6.18 3.25L7 14.14 2 9.27l6.91-1.01L12 2z" />
</svg>
<span className="text-white font-medium">66</span>
<span className="text-white font-medium">{stats.stars}</span>
</a>
<span className="text-zinc-800 hidden sm:inline">·</span>
<a
+1 -1
View File
@@ -13,7 +13,7 @@ const COMMAND_GROUPS: CommandGroup[] = [
{ cmd: 'install <repo>', desc: 'Install from GitHub' },
{ cmd: 'add <repo>', desc: 'Alias for install' },
{ cmd: 'recommend', desc: 'Smart suggestions' },
{ cmd: 'find <query>', desc: 'Search 15K+ skills' },
{ cmd: 'tree', desc: 'Browse skill taxonomy' },
{ cmd: 'marketplace', desc: 'Browse skills' },
],
},
+9
View File
@@ -43,6 +43,15 @@ const FEATURES: Feature[] = [
</svg>
)
},
{
title: 'Skill Tree',
description: 'Browse 15K+ skills in a hierarchical taxonomy with 12 categories.',
icon: (
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M3 7v10a2 2 0 002 2h14a2 2 0 002-2V9a2 2 0 00-2-2h-6l-2-2H5a2 2 0 00-2 2z" />
</svg>
)
},
{
title: 'Workflows',
description: 'Compose multi-step automated skill sequences.',
+6 -2
View File
@@ -1,6 +1,10 @@
import React, { useState, useEffect } from 'react';
import { Button } from './Button';
interface HeroProps {
version: string;
}
const ASCII_LOGO = `
███████╗██╗ ██╗██╗██╗ ██╗ ██╗ ██╗██╗████████╗
██╔════╝██║ ██╔╝██║██║ ██║ ██║ ██╔╝██║╚══██╔══╝
@@ -55,7 +59,7 @@ const COPY_ICON = (
</svg>
);
export function Hero(): React.ReactElement {
export function Hero({ version }: HeroProps): React.ReactElement {
const [copied, setCopied] = useState(false);
const [visibleLines, setVisibleLines] = useState(0);
const [typingIndex, setTypingIndex] = useState(0);
@@ -126,7 +130,7 @@ export function Hero(): React.ReactElement {
<div className="animate-fade-in">
<div className="inline-flex items-center space-x-2 border border-zinc-800 bg-zinc-900/50 px-2 py-0.5 mb-3 backdrop-blur-sm">
<span className="flex h-1.5 w-1.5 bg-white rounded-full"></span>
<span className="text-xs font-mono text-zinc-400">v1.8.0</span>
<span className="text-xs font-mono text-zinc-400">v{version}</span>
</div>
<h1 className="text-2xl sm:text-3xl lg:text-4xl font-bold tracking-tight text-white mb-3 font-mono">
+119
View File
@@ -0,0 +1,119 @@
import { useState, useEffect } from 'react';
interface Stats {
version: string;
downloads: string;
stars: number;
loading: boolean;
}
const CACHE_KEY = 'skillkit_stats_cache';
const CACHE_TTL = 5 * 60 * 1000; // 5 minutes
interface CachedStats {
data: Omit<Stats, 'loading'>;
timestamp: number;
}
function formatDownloads(count: number): string {
if (count >= 1000000) {
return `${(count / 1000000).toFixed(1)}M`;
}
if (count >= 1000) {
return `${(count / 1000).toFixed(1)}k`;
}
return count.toString();
}
function getCachedStats(): CachedStats | null {
try {
const cached = localStorage.getItem(CACHE_KEY);
if (cached) {
const parsed: CachedStats = JSON.parse(cached);
if (Date.now() - parsed.timestamp < CACHE_TTL) {
return parsed;
}
}
} catch {
// Ignore localStorage errors
}
return null;
}
function setCachedStats(data: Omit<Stats, 'loading'>): void {
try {
const cached: CachedStats = {
data,
timestamp: Date.now(),
};
localStorage.setItem(CACHE_KEY, JSON.stringify(cached));
} catch {
// Ignore localStorage errors
}
}
export function useStats(): Stats {
const [stats, setStats] = useState<Stats>({
version: '1.9.0',
downloads: '2.4k',
stars: 66,
loading: true,
});
useEffect(() => {
const cached = getCachedStats();
if (cached) {
setStats({ ...cached.data, loading: false });
return;
}
async function fetchStats(): Promise<void> {
try {
const [npmResponse, githubResponse] = await Promise.allSettled([
fetch('https://api.npmjs.org/downloads/point/last-month/skillkit'),
fetch('https://api.github.com/repos/rohitg00/skillkit'),
]);
let downloads = '2.4k';
let stars = 66;
let version = '1.9.0';
if (npmResponse.status === 'fulfilled' && npmResponse.value.ok) {
const npmData = await npmResponse.value.json();
if (typeof npmData.downloads === 'number' && Number.isFinite(npmData.downloads)) {
downloads = formatDownloads(npmData.downloads);
}
}
if (githubResponse.status === 'fulfilled' && githubResponse.value.ok) {
const githubData = await githubResponse.value.json();
if (typeof githubData.stargazers_count === 'number' && Number.isFinite(githubData.stargazers_count)) {
stars = githubData.stargazers_count;
}
}
try {
const registryResponse = await fetch('https://registry.npmjs.org/skillkit/latest');
if (registryResponse.ok) {
const registryData = await registryResponse.json();
if (registryData.version) {
version = registryData.version;
}
}
} catch {
// Use default version
}
const newStats = { version, downloads, stars };
setCachedStats(newStats);
setStats({ ...newStats, loading: false });
} catch {
setStats((prev) => ({ ...prev, loading: false }));
}
}
fetchStats();
}, []);
return stats;
}
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "skillkit",
"private": true,
"version": "1.8.0",
"version": "1.10.0",
"type": "module",
"scripts": {
"dev": "vite",
+19 -2
View File
@@ -1,8 +1,8 @@
{
"$schema": "./schema.json",
"version": 1,
"updatedAt": "2026-01-26",
"totalSkills": 15064,
"updatedAt": "2026-02-02",
"totalSkills": 15065,
"curatedCollections": 31,
"skills": [
{
@@ -33112,6 +33112,23 @@
"k8s"
]
},
{
"id": "rohitg00/beautiful-mermaid/beautiful-mermaid",
"name": "Beautiful Mermaid",
"description": "Render Mermaid diagrams as beautiful SVGs or ASCII art. 5 diagram types, 15 themes, Shiki integration, zero DOM dependencies.",
"source": "rohitg00/beautiful-mermaid",
"tags": [
"mermaid",
"diagrams",
"visualization",
"flowchart",
"sequence",
"uml",
"ascii",
"svg",
"themes"
]
},
{
"id": "itsmostafa/aws-agent-skills/cognito",
"name": "Cognito",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "skillkit-monorepo",
"version": "1.8.0",
"version": "1.10.0",
"private": true,
"description": "Universal skills loader for AI coding agents - Monorepo",
"type": "module",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/agents",
"version": "1.8.1",
"version": "1.10.0",
"description": "Agent adapters for SkillKit - supports 32 AI coding agents",
"type": "module",
"main": "./dist/index.js",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/cli",
"version": "1.8.1",
"version": "1.10.0",
"description": "CLI commands for SkillKit",
"type": "module",
"main": "./dist/index.js",
+187 -22
View File
@@ -7,7 +7,7 @@
import chalk from 'chalk';
import { Command, Option } from 'clipanion';
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { join, basename } from 'node:path';
import { homedir } from 'node:os';
import {
findAllAgents,
@@ -17,8 +17,14 @@ import {
validateAgent,
translateAgent,
getAgentTargetDirectory,
discoverSkills,
readSkillContent,
generateSubagentFromSkill,
type CustomAgent,
type AgentType,
type Skill,
type AgentPermissionMode,
type SkillToSubagentOptions,
} from '@skillkit/core';
import {
getBundledAgents,
@@ -28,7 +34,6 @@ import {
isAgentInstalled,
type BundledAgent,
} from '@skillkit/resources';
// Agent discovery uses root directories, not skill directories
export class AgentCommand extends Command {
static override paths = [['agent']];
@@ -43,17 +48,19 @@ export class AgentCommand extends Command {
that can be invoked with @mentions or the --agent flag.
Sub-commands:
agent list - List all installed agents
agent show - Show agent details
agent create - Create a new agent
agent translate - Translate agents between formats
agent sync - Sync agents to target AI agent
agent validate - Validate agent definitions
agent list - List all installed agents
agent show - Show agent details
agent create - Create a new agent
agent from-skill - Convert a skill to a subagent
agent translate - Translate agents between formats
agent sync - Sync agents to target AI agent
agent validate - Validate agent definitions
`,
examples: [
['List all agents', '$0 agent list'],
['Show agent details', '$0 agent show architect'],
['Create new agent', '$0 agent create security-reviewer'],
['Convert skill to subagent', '$0 agent from-skill code-simplifier'],
['Translate to Cursor format', '$0 agent translate --to cursor'],
['Sync agents', '$0 agent sync --agent claude-code'],
],
@@ -64,6 +71,7 @@ export class AgentCommand extends Command {
console.log(' agent list List all installed agents');
console.log(' agent show <name> Show agent details');
console.log(' agent create <name> Create a new agent');
console.log(' agent from-skill <name> Convert a skill to a subagent');
console.log(' agent translate Translate agents between formats');
console.log(' agent sync Sync agents to target AI agent');
console.log(' agent validate [path] Validate agent definitions');
@@ -254,7 +262,6 @@ export class AgentCreateCommand extends Command {
});
async execute(): Promise<number> {
// Validate agent name format
const namePattern = /^[a-z0-9]+(-[a-z0-9]+)*$/;
if (!namePattern.test(this.name)) {
console.log(chalk.red('Invalid agent name: must be lowercase alphanumeric with hyphens'));
@@ -262,7 +269,6 @@ export class AgentCreateCommand extends Command {
return 1;
}
// Determine target directory
let targetDir: string;
if (this.global) {
targetDir = join(homedir(), '.claude', 'agents');
@@ -270,23 +276,19 @@ export class AgentCreateCommand extends Command {
targetDir = join(process.cwd(), '.claude', 'agents');
}
// Create directory if needed
if (!existsSync(targetDir)) {
mkdirSync(targetDir, { recursive: true });
}
// Check if agent already exists
const agentPath = join(targetDir, `${this.name}.md`);
if (existsSync(agentPath)) {
console.log(chalk.red(`Agent already exists: ${agentPath}`));
return 1;
}
// Generate content
const description = this.description || `${this.name} agent`;
const content = generateAgentTemplate(this.name, description, this.model);
// Write file
writeFileSync(agentPath, content);
console.log(chalk.green(`Created agent: ${agentPath}`));
@@ -350,11 +352,9 @@ export class AgentTranslateCommand extends Command {
const searchDirs = [process.cwd()];
const targetAgent = this.to as AgentType;
// Get agents to translate
let agents: CustomAgent[];
if (this.source) {
// Translate from custom source path
const sourcePath = this.source.startsWith('/')
? this.source
: join(process.cwd(), this.source);
@@ -391,7 +391,6 @@ export class AgentTranslateCommand extends Command {
return 0;
}
// Determine output directory
const outputDir = this.output || getAgentTargetDirectory(process.cwd(), targetAgent);
console.log(chalk.cyan(`Translating ${agents.length} agent(s) to ${targetAgent} format...\n`));
@@ -424,7 +423,6 @@ export class AgentTranslateCommand extends Command {
}
}
} else {
// Create directory if needed
if (!existsSync(outputDir)) {
mkdirSync(outputDir, { recursive: true });
}
@@ -529,12 +527,10 @@ export class AgentValidateCommand extends Command {
let hasErrors = false;
if (this.agentPath) {
// Validate specific path
const result = validateAgent(this.agentPath);
printValidationResult(this.agentPath, result);
hasErrors = !result.valid;
} else if (this.all) {
// Validate all agents
const searchDirs = [process.cwd()];
const agents = findAllAgents(searchDirs);
@@ -559,8 +555,6 @@ export class AgentValidateCommand extends Command {
}
}
// Helper functions
function printAgent(agent: CustomAgent): void {
const status = agent.enabled ? chalk.green('✓') : chalk.red('○');
const name = agent.enabled ? agent.name : chalk.dim(agent.name);
@@ -831,9 +825,180 @@ export class AgentAvailableCommand extends Command {
}
}
export class AgentFromSkillCommand extends Command {
static override paths = [['agent', 'from-skill']];
static override usage = Command.Usage({
description: 'Convert a skill into a Claude Code subagent',
details: `
Converts a SkillKit skill into a Claude Code native subagent format.
The generated .md file can be used with @mentions in Claude Code.
By default, the subagent references the skill (skills: [skill-name]).
Use --inline to embed the full skill content in the system prompt.
`,
examples: [
['Convert skill to subagent', '$0 agent from-skill code-simplifier'],
['Create global subagent', '$0 agent from-skill code-simplifier --global'],
['Embed skill content inline', '$0 agent from-skill code-simplifier --inline'],
['Set model for subagent', '$0 agent from-skill code-simplifier --model opus'],
['Preview without writing', '$0 agent from-skill code-simplifier --dry-run'],
],
});
skillName = Option.String({ required: true });
inline = Option.Boolean('--inline,-i', false, {
description: 'Embed full skill content in system prompt',
});
model = Option.String('--model,-m', {
description: 'Model to use (sonnet, opus, haiku, inherit)',
});
permission = Option.String('--permission,-p', {
description: 'Permission mode (default, plan, auto-edit, full-auto, bypassPermissions)',
});
global = Option.Boolean('--global,-g', false, {
description: 'Create in ~/.claude/agents/ instead of .claude/agents/',
});
output = Option.String('--output,-o', {
description: 'Custom output filename (without .md)',
});
dryRun = Option.Boolean('--dry-run,-n', false, {
description: 'Preview without writing files',
});
async execute(): Promise<number> {
const skills = discoverSkills(process.cwd());
const skill = skills.find((s: Skill) => s.name === this.skillName);
if (!skill) {
console.log(chalk.red(`Skill not found: ${this.skillName}`));
console.log(chalk.dim('Available skills:'));
for (const s of skills.slice(0, 10)) {
console.log(chalk.dim(` - ${s.name}`));
}
if (skills.length > 10) {
console.log(chalk.dim(` ... and ${skills.length - 10} more`));
}
return 1;
}
const skillContent = readSkillContent(skill.path);
if (!skillContent) {
console.log(chalk.red(`Could not read skill content: ${skill.path}`));
return 1;
}
const options: SkillToSubagentOptions = {
inline: this.inline,
};
if (this.model) {
const validModels = ['sonnet', 'opus', 'haiku', 'inherit'];
if (!validModels.includes(this.model)) {
console.log(chalk.red(`Invalid model: ${this.model}`));
console.log(chalk.dim(`Valid options: ${validModels.join(', ')}`));
return 1;
}
options.model = this.model as 'sonnet' | 'opus' | 'haiku' | 'inherit';
}
if (this.permission) {
const validModes = ['default', 'plan', 'auto-edit', 'full-auto', 'bypassPermissions'];
if (!validModes.includes(this.permission)) {
console.log(chalk.red(`Invalid permission mode: ${this.permission}`));
console.log(chalk.dim(`Valid options: ${validModes.join(', ')}`));
return 1;
}
options.permissionMode = this.permission as AgentPermissionMode;
}
const content = generateSubagentFromSkill(skill, skillContent, options);
const targetDir = this.global
? join(homedir(), '.claude', 'agents')
: join(process.cwd(), '.claude', 'agents');
let filename: string;
if (this.output) {
const sanitized = sanitizeFilename(this.output);
if (!sanitized) {
console.log(chalk.red(`Invalid output filename: ${this.output}`));
console.log(chalk.dim('Filename must contain only alphanumeric characters, hyphens, and underscores'));
return 1;
}
filename = `${sanitized}.md`;
} else {
const sanitized = sanitizeFilename(skill.name);
if (!sanitized) {
console.log(chalk.red(`Invalid skill name for filename: ${skill.name}`));
console.log(chalk.dim('Skill name must contain only alphanumeric characters, hyphens, and underscores'));
return 1;
}
filename = `${sanitized}.md`;
}
const outputPath = join(targetDir, filename);
if (this.dryRun) {
console.log(chalk.cyan('Preview (dry run):\n'));
console.log(chalk.dim(`Would write to: ${outputPath}`));
console.log(chalk.dim('─'.repeat(50)));
console.log(content);
console.log(chalk.dim('─'.repeat(50)));
return 0;
}
if (!existsSync(targetDir)) {
mkdirSync(targetDir, { recursive: true });
}
if (existsSync(outputPath)) {
console.log(chalk.yellow(`Overwriting existing file: ${outputPath}`));
}
writeFileSync(outputPath, content);
console.log(chalk.green(`Created subagent: ${outputPath}`));
console.log();
console.log(chalk.dim(`Invoke with: @${skill.name}`));
if (!this.inline) {
console.log(chalk.dim(`Skills referenced: ${skill.name}`));
} else {
console.log(chalk.dim('Skill content embedded inline'));
}
return 0;
}
}
function formatCategoryName(category: string): string {
return category
.split('-')
.map(word => word.charAt(0).toUpperCase() + word.slice(1))
.join(' ');
}
function sanitizeFilename(input: string): string | null {
const base = basename(input);
const stem = base.replace(/\.md$/i, '');
if (!stem || stem.startsWith('.') || stem.startsWith('-')) {
return null;
}
if (!/^[a-zA-Z0-9][a-zA-Z0-9_-]*$/.test(stem)) {
return null;
}
if (stem.length > 64) {
return null;
}
return stem;
}
+5 -1
View File
@@ -43,12 +43,13 @@ export { PlanCommand } from './plan.js';
export { CommandCmd, CommandAvailableCommand, CommandInstallCommand } from './command.js';
export { AICommand } from './ai.js';
export { AuditCommand } from './audit.js';
export { PublishCommand } from './publish.js';
export { PublishCommand, PublishSubmitCommand } from './publish.js';
export {
AgentCommand,
AgentListCommand,
AgentShowCommand,
AgentCreateCommand,
AgentFromSkillCommand,
AgentTranslateCommand,
AgentSyncCommand,
AgentValidateCommand,
@@ -104,3 +105,6 @@ export {
GuidelineCreateCommand,
GuidelineRemoveCommand,
} from './guideline.js';
// Tree command for hierarchical skill browsing (Phase 21)
export { TreeCommand } from './tree.js';
+23 -3
View File
@@ -1,5 +1,9 @@
import { existsSync, mkdirSync, cpSync, rmSync, symlinkSync } from 'node:fs';
import { join } from 'node:path';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
const execFileAsync = promisify(execFile);
import { Command, Option } from 'clipanion';
import { detectProvider, isLocalPath, getProvider, evaluateSkillDirectory } from '@skillkit/core';
import type { SkillMetadata, GitProvider, AgentType } from '@skillkit/core';
@@ -42,7 +46,8 @@ export class InstallCommand extends Command {
['Install from GitHub', '$0 install owner/repo'],
['Install from GitLab', '$0 install gitlab:owner/repo'],
['Install from Bitbucket', '$0 install bitbucket:owner/repo'],
['Install specific skills (CI/CD)', '$0 install owner/repo --skills=pdf,xlsx'],
['Install specific skill', '$0 install owner/repo --skill=pdf'],
['Install multiple skills (CI/CD)', '$0 install owner/repo --skills=pdf,xlsx'],
['Install all skills non-interactively', '$0 install owner/repo --all'],
['Install from local path', '$0 install ./my-skills'],
['Install globally', '$0 install owner/repo --global'],
@@ -53,7 +58,7 @@ export class InstallCommand extends Command {
source = Option.String({ required: true });
skills = Option.String('--skills,-s', {
skills = Option.String('--skills,--skill,-s', {
description: 'Comma-separated list of skills to install (non-interactive)',
});
@@ -143,7 +148,7 @@ export class InstallCommand extends Command {
}
console.log('');
console.log(colors.muted(`Total: ${discoveredSkills.length} skill(s)`));
console.log(colors.muted('To install: skillkit install <source> --skills=skill1,skill2'));
console.log(colors.muted('To install: skillkit install <source> --skill=name'));
}
const cleanupPath = result.tempRoot || result.path;
@@ -334,6 +339,21 @@ export class InstallCommand extends Command {
if (isSymlinkMode && primaryPath === null) {
primaryPath = targetPath;
}
// Auto-install npm dependencies if package.json exists
const packageJsonPath = join(targetPath, 'package.json');
if (existsSync(packageJsonPath)) {
s.stop(`Installed ${skillName} to ${adapter.name}`);
s.start(`Installing npm dependencies for ${skillName}...`);
try {
await execFileAsync('npm', ['install', '--production'], { cwd: targetPath });
s.stop(`Installed dependencies for ${skillName}`);
} catch (npmErr) {
s.stop(colors.warning(`Dependencies failed for ${skillName}`));
console.log(colors.muted('Run manually: npm install in ' + targetPath));
}
s.start(`Finishing ${skillName} installation...`);
}
}
const metadata: SkillMetadata = {
+336 -126
View File
@@ -1,8 +1,20 @@
import { existsSync, readFileSync } from 'node:fs';
import { join, basename, dirname } from 'node:path';
import { execSync } from 'node:child_process';
import { existsSync, readFileSync, mkdirSync, writeFileSync, readdirSync, statSync } from 'node:fs';
import { join, basename, dirname, resolve } from 'node:path';
import chalk from 'chalk';
import { Command, Option } from 'clipanion';
import { generateWellKnownIndex, type WellKnownSkill } from '@skillkit/core';
function sanitizeSkillName(name: string): string | null {
if (!name || typeof name !== 'string') return null;
const base = basename(name);
if (base !== name || name.includes('..') || name.includes('/') || name.includes('\\')) {
return null;
}
if (!/^[a-zA-Z0-9._-]+$/.test(name)) {
return null;
}
return name;
}
interface SkillFrontmatter {
name?: string;
@@ -15,141 +27,201 @@ export class PublishCommand extends Command {
static override paths = [['publish']];
static override usage = Command.Usage({
description: 'Publish your skill to the SkillKit marketplace',
description: 'Generate well-known skills structure for hosting',
details: `
This command generates the RFC 8615 well-known URI structure for hosting skills.
The output includes:
- .well-known/skills/index.json - Skill manifest for auto-discovery
- .well-known/skills/{skill-name}/SKILL.md - Individual skill files
Users can then install skills via: skillkit add https://your-domain.com
`,
examples: [
['Publish skill from current directory', '$0 publish'],
['Publish skill from specific path', '$0 publish ./my-skill'],
['Publish with custom name', '$0 publish --name my-awesome-skill'],
['Generate from current directory', '$0 publish'],
['Generate from specific path', '$0 publish ./my-skills'],
['Generate to custom output directory', '$0 publish --output ./public'],
['Preview without writing', '$0 publish --dry-run'],
],
});
skillPath = Option.String({ required: false, name: 'path' });
name = Option.String('--name,-n', {
description: 'Custom skill name (default: parsed from SKILL.md)',
output = Option.String('--output,-o', {
description: 'Output directory for well-known structure (default: current directory)',
});
dryRun = Option.Boolean('--dry-run', false, {
description: 'Show what would be published without actually publishing',
dryRun = Option.Boolean('--dry-run,-n', false, {
description: 'Show what would be generated without writing files',
});
async execute(): Promise<number> {
const skillPath = this.skillPath || process.cwd();
const skillMdPath = this.findSkillMd(skillPath);
const basePath = this.skillPath || process.cwd();
const outputDir = this.output || basePath;
if (!skillMdPath) {
console.error(chalk.red('No SKILL.md found'));
console.error(chalk.dim('Run this command from a directory containing SKILL.md'));
console.error(chalk.dim('Or specify the path: skillkit publish ./path/to/skill'));
console.log(chalk.cyan('Generating well-known skills structure...\n'));
const discoveredSkills = this.discoverSkills(basePath);
if (discoveredSkills.length === 0) {
console.error(chalk.red('No skills found'));
console.error(chalk.dim('Skills must contain a SKILL.md file with frontmatter'));
return 1;
}
console.log(chalk.cyan('Publishing skill to SkillKit marketplace...\n'));
console.log(chalk.white(`Found ${discoveredSkills.length} skill(s):\n`));
// Parse SKILL.md
const content = readFileSync(skillMdPath, 'utf-8');
const frontmatter = this.parseFrontmatter(content);
const skillName = this.name || frontmatter.name || basename(dirname(skillMdPath));
const wellKnownSkills: WellKnownSkill[] = [];
// Get git repo info
const repoInfo = this.getRepoInfo(dirname(skillMdPath));
if (!repoInfo) {
console.error(chalk.red('Not a git repository or no remote configured'));
console.error(chalk.dim('Your skill must be in a git repository with a GitHub remote'));
const validSkills: Array<{ name: string; safeName: string; description?: string; path: string }> = [];
for (const skill of discoveredSkills) {
const safeName = sanitizeSkillName(skill.name);
if (!safeName) {
console.log(chalk.yellow(` ${chalk.yellow('⚠')} Skipping "${skill.name}" (invalid name - must be alphanumeric with hyphens/underscores)`));
continue;
}
const files = this.getSkillFiles(skill.path);
console.log(chalk.dim(` ${chalk.green('●')} ${safeName}`));
console.log(chalk.dim(` Description: ${skill.description || 'No description'}`));
console.log(chalk.dim(` Files: ${files.join(', ')}`));
validSkills.push({ name: skill.name, safeName, description: skill.description, path: skill.path });
wellKnownSkills.push({
name: safeName,
description: skill.description,
files,
});
}
if (validSkills.length === 0) {
console.error(chalk.red('\nNo valid skills to publish'));
return 1;
}
// Build skill entry
const skillSlug = this.slugify(skillName);
if (!skillSlug) {
console.error(chalk.red('Skill name produces an empty slug.'));
console.error(chalk.dim('Please pass --name with letters or numbers.'));
return 1;
}
const skillEntry = {
id: `${repoInfo.owner}/${repoInfo.repo}/${skillSlug}`,
name: this.formatName(skillName),
description: frontmatter.description || `Best practices and patterns for ${this.formatName(skillName)}`,
source: `${repoInfo.owner}/${repoInfo.repo}`,
tags: frontmatter.tags || this.inferTags(skillName, frontmatter.description || ''),
};
console.log(chalk.white('Skill details:'));
console.log(chalk.dim(` ID: ${skillEntry.id}`));
console.log(chalk.dim(` Name: ${skillEntry.name}`));
console.log(chalk.dim(` Description: ${skillEntry.description}`));
console.log(chalk.dim(` Source: ${skillEntry.source}`));
console.log(chalk.dim(` Tags: ${skillEntry.tags.join(', ')}`));
console.log();
console.log('');
if (this.dryRun) {
console.log(chalk.yellow('Dry run - not publishing'));
console.log(chalk.dim('JSON entry that would be added:'));
console.log(JSON.stringify(skillEntry, null, 2));
console.log(chalk.yellow('Dry run - not writing files\n'));
console.log(chalk.white('Would generate:'));
console.log(chalk.dim(` ${outputDir}/.well-known/skills/index.json`));
for (const skill of wellKnownSkills) {
for (const file of skill.files) {
console.log(chalk.dim(` ${outputDir}/.well-known/skills/${skill.name}/${file}`));
}
}
console.log('');
console.log(chalk.white('index.json preview:'));
console.log(JSON.stringify(generateWellKnownIndex(wellKnownSkills), null, 2));
return 0;
}
// Create GitHub issue to add skill
const issueBody = this.createIssueBody(skillEntry);
const issueTitle = encodeURIComponent(`[Publish] ${skillEntry.name}`);
const issueBodyEncoded = encodeURIComponent(issueBody);
const issueUrl = `https://github.com/rohitg00/skillkit/issues/new?title=${issueTitle}&body=${issueBodyEncoded}&labels=skill-submission,publish`;
const wellKnownDir = join(outputDir, '.well-known', 'skills');
mkdirSync(wellKnownDir, { recursive: true });
console.log(chalk.green('Opening GitHub to submit your skill...\n'));
for (const skill of validSkills) {
const skillDir = join(wellKnownDir, skill.safeName);
const resolvedSkillDir = resolve(skillDir);
const resolvedWellKnownDir = resolve(wellKnownDir);
try {
// Try to open the URL
const openCmd =
process.platform === 'darwin'
? `open "${issueUrl}"`
: process.platform === 'win32'
? `cmd /c start "" "${issueUrl}"`
: `xdg-open "${issueUrl}"`;
execSync(openCmd, { stdio: 'ignore' });
if (!resolvedSkillDir.startsWith(resolvedWellKnownDir)) {
console.log(chalk.yellow(` Skipping "${skill.name}" (path traversal detected)`));
continue;
}
console.log(chalk.green('GitHub issue page opened!'));
console.log(chalk.dim('Review and submit the issue to publish your skill.'));
} catch {
console.log(chalk.yellow('Could not open browser automatically.'));
console.log(chalk.dim('Please open this URL manually:\n'));
console.log(chalk.cyan(issueUrl));
mkdirSync(skillDir, { recursive: true });
const files = this.getSkillFiles(skill.path);
for (const file of files) {
const safeFile = basename(file);
const sourcePath = join(skill.path, file);
const destPath = join(skillDir, safeFile);
const content = readFileSync(sourcePath, 'utf-8');
writeFileSync(destPath, content);
}
}
console.log();
const index = generateWellKnownIndex(wellKnownSkills);
writeFileSync(join(wellKnownDir, 'index.json'), JSON.stringify(index, null, 2));
console.log(chalk.green('Generated well-known structure:\n'));
console.log(chalk.dim(` ${wellKnownDir}/index.json`));
for (const skill of wellKnownSkills) {
console.log(chalk.dim(` ${wellKnownDir}/${skill.name}/`));
}
console.log('');
console.log(chalk.cyan('Next steps:'));
console.log(chalk.dim(' 1. Review the skill details in the GitHub issue'));
console.log(chalk.dim(' 2. Submit the issue'));
console.log(chalk.dim(' 3. A maintainer will review and add your skill'));
console.log(chalk.dim(' 1. Deploy the .well-known directory to your web server'));
console.log(chalk.dim(' 2. Users can install via: skillkit add https://your-domain.com'));
console.log(chalk.dim(' 3. Skills auto-discovered from /.well-known/skills/index.json'));
return 0;
}
private findSkillMd(basePath: string): string | null {
// Check if path is directly to SKILL.md
if (basePath.endsWith('SKILL.md') && existsSync(basePath)) {
return basePath;
private discoverSkills(basePath: string): Array<{ name: string; description?: string; path: string }> {
const skills: Array<{ name: string; description?: string; path: string }> = [];
const skillMdPath = join(basePath, 'SKILL.md');
if (existsSync(skillMdPath)) {
const content = readFileSync(skillMdPath, 'utf-8');
const frontmatter = this.parseFrontmatter(content);
skills.push({
name: frontmatter.name || basename(basePath),
description: frontmatter.description,
path: basePath,
});
return skills;
}
// Check if SKILL.md exists in the directory
const direct = join(basePath, 'SKILL.md');
if (existsSync(direct)) {
return direct;
}
// Check common skill locations
const locations = [
join(basePath, 'skills', 'SKILL.md'),
join(basePath, '.claude', 'skills', 'SKILL.md'),
join(basePath, '.cursor', 'skills', 'SKILL.md'),
const searchDirs = [
basePath,
join(basePath, 'skills'),
join(basePath, '.claude', 'skills'),
];
for (const loc of locations) {
if (existsSync(loc)) {
return loc;
for (const searchDir of searchDirs) {
if (!existsSync(searchDir)) continue;
const entries = readdirSync(searchDir);
for (const entry of entries) {
const entryPath = join(searchDir, entry);
if (!statSync(entryPath).isDirectory()) continue;
const entrySkillMd = join(entryPath, 'SKILL.md');
if (existsSync(entrySkillMd)) {
const content = readFileSync(entrySkillMd, 'utf-8');
const frontmatter = this.parseFrontmatter(content);
skills.push({
name: frontmatter.name || entry,
description: frontmatter.description,
path: entryPath,
});
}
}
}
return null;
return skills;
}
private getSkillFiles(skillPath: string): string[] {
const files: string[] = [];
const entries = readdirSync(skillPath);
for (const entry of entries) {
const entryPath = join(skillPath, entry);
if (statSync(entryPath).isFile()) {
if (entry.startsWith('.') || entry === '.skillkit-metadata.json') continue;
files.push(entry);
}
}
if (!files.includes('SKILL.md')) {
files.unshift('SKILL.md');
}
return files;
}
private parseFrontmatter(content: string): SkillFrontmatter {
@@ -161,7 +233,6 @@ export class PublishCommand extends Command {
let inTagsList = false;
for (const line of lines) {
// Handle multiline tags list
if (inTagsList) {
const tagMatch = line.match(/^\s*-\s*(.+)$/);
if (tagMatch) {
@@ -190,7 +261,6 @@ export class PublishCommand extends Command {
frontmatter.version = value.replace(/^["']|["']$/g, '');
break;
case 'tags':
// Parse YAML array: [tag1, tag2] or multiline list
if (value.startsWith('[')) {
frontmatter.tags = value
.slice(1, -1)
@@ -207,6 +277,171 @@ export class PublishCommand extends Command {
return frontmatter;
}
}
export class PublishSubmitCommand extends Command {
static override paths = [['publish', 'submit']];
static override usage = Command.Usage({
description: 'Submit skill to SkillKit marketplace (requires review)',
examples: [
['Submit skill from current directory', '$0 publish submit'],
['Submit with custom name', '$0 publish submit --name my-skill'],
],
});
skillPath = Option.String({ required: false, name: 'path' });
name = Option.String('--name,-n', {
description: 'Custom skill name',
});
dryRun = Option.Boolean('--dry-run', false, {
description: 'Show what would be submitted',
});
async execute(): Promise<number> {
const skillPath = this.skillPath || process.cwd();
const skillMdPath = this.findSkillMd(skillPath);
if (!skillMdPath) {
console.error(chalk.red('No SKILL.md found'));
console.error(chalk.dim('Run this command from a directory containing SKILL.md'));
return 1;
}
console.log(chalk.cyan('Submitting skill to SkillKit marketplace...\n'));
const content = readFileSync(skillMdPath, 'utf-8');
const frontmatter = this.parseFrontmatter(content);
const skillName = this.name || frontmatter.name || basename(dirname(skillMdPath));
const repoInfo = this.getRepoInfo(dirname(skillMdPath));
if (!repoInfo) {
console.error(chalk.red('Not a git repository or no remote configured'));
console.error(chalk.dim('Your skill must be in a git repository with a GitHub remote'));
return 1;
}
const skillSlug = this.slugify(skillName);
if (!skillSlug) {
console.error(chalk.red('Skill name produces an empty slug.'));
console.error(chalk.dim('Please pass --name with letters or numbers.'));
return 1;
}
const skillEntry = {
id: `${repoInfo.owner}/${repoInfo.repo}/${skillSlug}`,
name: this.formatName(skillName),
description: frontmatter.description || `Best practices for ${this.formatName(skillName)}`,
source: `${repoInfo.owner}/${repoInfo.repo}`,
tags: frontmatter.tags || ['general'],
};
console.log(chalk.white('Skill details:'));
console.log(chalk.dim(` ID: ${skillEntry.id}`));
console.log(chalk.dim(` Name: ${skillEntry.name}`));
console.log(chalk.dim(` Description: ${skillEntry.description}`));
console.log(chalk.dim(` Source: ${skillEntry.source}`));
console.log(chalk.dim(` Tags: ${skillEntry.tags.join(', ')}`));
console.log();
if (this.dryRun) {
console.log(chalk.yellow('Dry run - not submitting'));
console.log(JSON.stringify(skillEntry, null, 2));
return 0;
}
const issueBody = this.createIssueBody(skillEntry);
const issueTitle = encodeURIComponent(`[Publish] ${skillEntry.name}`);
const issueBodyEncoded = encodeURIComponent(issueBody);
const issueUrl = `https://github.com/rohitg00/skillkit/issues/new?title=${issueTitle}&body=${issueBodyEncoded}&labels=skill-submission,publish`;
console.log(chalk.green('Opening GitHub to submit your skill...\n'));
try {
const { execSync } = await import('node:child_process');
const openCmd =
process.platform === 'darwin'
? `open "${issueUrl}"`
: process.platform === 'win32'
? `cmd /c start "" "${issueUrl}"`
: `xdg-open "${issueUrl}"`;
execSync(openCmd, { stdio: 'ignore' });
console.log(chalk.green('GitHub issue page opened!'));
console.log(chalk.dim('Review and submit the issue.'));
} catch {
console.log(chalk.yellow('Could not open browser automatically.'));
console.log(chalk.dim('Please open this URL manually:\n'));
console.log(chalk.cyan(issueUrl));
}
return 0;
}
private findSkillMd(basePath: string): string | null {
if (basePath.endsWith('SKILL.md') && existsSync(basePath)) {
return basePath;
}
const direct = join(basePath, 'SKILL.md');
if (existsSync(direct)) {
return direct;
}
const locations = [
join(basePath, 'skills', 'SKILL.md'),
join(basePath, '.claude', 'skills', 'SKILL.md'),
];
for (const loc of locations) {
if (existsSync(loc)) {
return loc;
}
}
return null;
}
private parseFrontmatter(content: string): SkillFrontmatter {
const match = content.match(/^---\r?\n([\s\S]*?)\r?\n---/);
if (!match) return {};
const frontmatter: SkillFrontmatter = {};
const lines = match[1].split(/\r?\n/);
for (const line of lines) {
const colonIdx = line.indexOf(':');
if (colonIdx === -1) continue;
const key = line.slice(0, colonIdx).trim();
const value = line.slice(colonIdx + 1).trim();
switch (key) {
case 'name':
frontmatter.name = value.replace(/^["']|["']$/g, '');
break;
case 'description':
frontmatter.description = value.replace(/^["']|["']$/g, '');
break;
case 'version':
frontmatter.version = value.replace(/^["']|["']$/g, '');
break;
case 'tags':
if (value.startsWith('[')) {
frontmatter.tags = value
.slice(1, -1)
.split(',')
.map(t => t.trim().replace(/^["']|["']$/g, ''))
.filter(t => t.length > 0);
}
break;
}
}
return frontmatter;
}
private slugify(name: string): string {
return name
@@ -218,19 +453,19 @@ export class PublishCommand extends Command {
private getRepoInfo(dir: string): { owner: string; repo: string } | null {
try {
const { execSync } = require('node:child_process');
const remote = execSync('git remote get-url origin', {
cwd: dir,
encoding: 'utf-8',
stdio: ['pipe', 'pipe', 'ignore'],
}).trim();
// Parse GitHub URL: git@github.com:owner/repo.git or https://github.com/owner/repo.git
const match = remote.match(/github\.com[:/]([^/]+)\/(.+?)(?:\.git)?$/);
if (match) {
return { owner: match[1], repo: match[2] };
}
} catch {
// Not a git repo or no remote
// Not a git repo
}
return null;
@@ -243,31 +478,6 @@ export class PublishCommand extends Command {
.join(' ');
}
private inferTags(name: string, description: string): string[] {
const tags: string[] = [];
const text = `${name} ${description}`.toLowerCase();
const tagMap: Record<string, string[]> = {
react: ['react', 'jsx', 'tsx'],
typescript: ['typescript', 'ts'],
nextjs: ['next', 'nextjs'],
testing: ['test', 'jest', 'vitest'],
mobile: ['mobile', 'react-native', 'expo'],
backend: ['backend', 'api', 'server'],
database: ['database', 'postgres', 'mysql', 'supabase'],
frontend: ['frontend', 'ui', 'design'],
devops: ['devops', 'ci', 'cd', 'docker'],
};
for (const [tag, keywords] of Object.entries(tagMap)) {
if (keywords.some(k => text.includes(k))) {
tags.push(tag);
}
}
return tags.length > 0 ? tags : ['general'];
}
private createIssueBody(
skill: { id: string; name: string; description: string; source: string; tags: string[] }
): string {
@@ -292,6 +502,6 @@ ${JSON.stringify(skill, null, 2)}
- [ ] Tags are appropriate
---
Submitted via \`skillkit publish\``;
Submitted via \`skillkit publish submit\``;
}
}
+332
View File
@@ -3,8 +3,10 @@ import { resolve } from 'node:path';
import {
type ProjectProfile,
type ScoredSkill,
type ExplainedScoredSkill,
ContextManager,
RecommendationEngine,
ReasoningRecommendationEngine,
buildSkillIndex,
saveIndex,
loadIndex as loadIndexFromCache,
@@ -50,6 +52,9 @@ export class RecommendCommand extends Command {
['Update skill index', '$0 recommend --update'],
['Search for skills by task', '$0 recommend --task "authentication"'],
['Search for skills (alias)', '$0 recommend --search "testing"'],
['Hybrid search (vector + keyword)', '$0 recommend --search "auth" --hybrid'],
['Hybrid search with query expansion', '$0 recommend --search "auth" --hybrid --expand'],
['Build hybrid search index', '$0 recommend --build-index'],
],
});
@@ -108,6 +113,41 @@ export class RecommendCommand extends Command {
description: 'Minimal output',
});
// Explain mode (reasoning-based recommendations)
explain = Option.Boolean('--explain,-e', false, {
description: 'Show detailed explanations for recommendations (uses reasoning engine)',
});
// Reasoning mode
reasoning = Option.Boolean('--reasoning,-r', false, {
description: 'Use LLM-based reasoning for recommendations',
});
// Show tree path
showPath = Option.Boolean('--show-path', false, {
description: 'Show category path for each recommendation',
});
// Hybrid search mode
hybrid = Option.Boolean('--hybrid,-H', false, {
description: 'Use hybrid search (vector + keyword)',
});
// Query expansion
expand = Option.Boolean('--expand,-x', false, {
description: 'Enable query expansion (requires --hybrid)',
});
// Reranking
rerank = Option.Boolean('--rerank', false, {
description: 'Enable LLM reranking (requires --hybrid)',
});
// Build index
buildIndex = Option.Boolean('--build-index', false, {
description: 'Build/rebuild the hybrid search embedding index',
});
async execute(): Promise<number> {
const targetPath = resolve(this.projectPath || process.cwd());
@@ -116,6 +156,16 @@ export class RecommendCommand extends Command {
return await this.updateIndex();
}
// Handle hybrid index building
if (this.buildIndex) {
return await this.buildHybridIndex();
}
// Validate hybrid-dependent options
if ((this.expand || this.rerank) && !this.hybrid) {
warn('--expand and --rerank require --hybrid flag. These options will be ignored.');
}
if (!this.quiet && !this.json) {
header('Skill Recommendations');
}
@@ -139,6 +189,11 @@ export class RecommendCommand extends Command {
return 0;
}
// Use reasoning engine if explain/reasoning flags are set
if (this.explain || this.reasoning || this.showPath) {
return await this.handleReasoningRecommendations(profile, index);
}
// Create recommendation engine
const engine = new RecommendationEngine();
engine.loadIndex(index);
@@ -146,6 +201,9 @@ export class RecommendCommand extends Command {
// Handle search mode (--search or --task)
const searchQuery = this.search || this.task;
if (searchQuery) {
if (this.hybrid) {
return await this.handleHybridSearch(engine, searchQuery);
}
return this.handleSearch(engine, searchQuery);
}
@@ -168,6 +226,63 @@ export class RecommendCommand extends Command {
return 0;
}
private async handleReasoningRecommendations(
profile: ProjectProfile,
index: { skills: ScoredSkill['skill'][]; sources: { name: string; url: string; lastFetched: string; skillCount: number }[]; version: number; lastUpdated: string }
): Promise<number> {
const s = !this.quiet && !this.json ? spinner() : null;
s?.start('Analyzing with reasoning engine...');
try {
const engine = new ReasoningRecommendationEngine();
engine.loadIndex(index);
await engine.initReasoning();
const result = await engine.recommendWithReasoning(profile, {
limit: this.limit ? parseInt(this.limit, 10) : 10,
minScore: this.minScore ? parseInt(this.minScore, 10) : 30,
categories: this.category,
excludeInstalled: !this.includeInstalled,
includeReasons: this.verbose,
reasoning: this.reasoning,
explainResults: this.explain,
useTree: true,
});
s?.stop('Analysis complete');
if (this.json) {
console.log(JSON.stringify(result, null, 2));
return 0;
}
this.displayExplainedRecommendations(
result.recommendations,
profile,
result.totalSkillsScanned,
result.reasoningSummary
);
return 0;
} catch (err) {
s?.stop(colors.error('Reasoning analysis failed'));
console.log(colors.muted(err instanceof Error ? err.message : String(err)));
console.log(colors.muted('Falling back to standard recommendations...'));
console.log('');
const engine = new RecommendationEngine();
engine.loadIndex(index);
const result = engine.recommend(profile, {
limit: this.limit ? parseInt(this.limit, 10) : 10,
minScore: this.minScore ? parseInt(this.minScore, 10) : 30,
categories: this.category,
excludeInstalled: !this.includeInstalled,
includeReasons: this.verbose,
});
this.displayRecommendations(result.recommendations, profile, result.totalSkillsScanned);
return 0;
}
}
private async getProjectProfile(projectPath: string): Promise<ProjectProfile | null> {
const manager = new ContextManager(projectPath);
let context = manager.get();
@@ -276,6 +391,223 @@ export class RecommendCommand extends Command {
console.log(colors.muted('Install with: skillkit install <source>'));
}
private displayExplainedRecommendations(
recommendations: ExplainedScoredSkill[],
profile: ProjectProfile,
totalScanned: number,
reasoningSummary?: string
): void {
this.showProjectProfile(profile);
if (reasoningSummary && !this.quiet) {
console.log(colors.dim('Reasoning: ') + colors.muted(reasoningSummary));
console.log('');
}
if (recommendations.length === 0) {
warn('No matching skills found.');
console.log(colors.muted('Try lowering the minimum score with --min-score'));
return;
}
console.log(colors.bold(`Explained Recommendations (${recommendations.length} of ${totalScanned} scanned):`));
console.log('');
for (const rec of recommendations) {
let scoreColor: (text: string) => string;
if (rec.score >= 70) {
scoreColor = colors.success;
} else if (rec.score >= 50) {
scoreColor = colors.warning;
} else {
scoreColor = colors.muted;
}
const scoreBar = progressBar(rec.score, 100, 10);
const qualityScore = rec.skill.quality ?? null;
const qualityDisplay = qualityScore !== null && qualityScore !== undefined
? ` ${formatQualityBadge(qualityScore)}`
: '';
console.log(` ${scoreColor(`${rec.score}%`)} ${colors.dim(scoreBar)} ${colors.bold(rec.skill.name)}${qualityDisplay}`);
if (this.showPath && rec.treePath && rec.treePath.length > 0) {
console.log(` ${colors.accent('Path:')} ${rec.treePath.join(' > ')}`);
}
if (rec.skill.description) {
console.log(` ${colors.muted(truncate(rec.skill.description, 70))}`);
}
if (rec.skill.source) {
console.log(` ${colors.dim('Source:')} ${rec.skill.source}`);
}
if (this.explain && rec.explanation) {
console.log(colors.dim(' Why this skill:'));
if (rec.explanation.matchedBecause.length > 0) {
console.log(` ${colors.success('├─')} Matched: ${rec.explanation.matchedBecause.join(', ')}`);
}
if (rec.explanation.relevantFor.length > 0) {
console.log(` ${colors.accent('├─')} Relevant for: ${rec.explanation.relevantFor.join(', ')}`);
}
if (rec.explanation.confidence) {
const confidenceColor = rec.explanation.confidence === 'high' ? colors.success :
rec.explanation.confidence === 'medium' ? colors.warning :
colors.muted;
console.log(` ${colors.dim('└─')} Confidence: ${confidenceColor(rec.explanation.confidence)}`);
}
}
if (this.verbose && rec.reasons.length > 0) {
console.log(colors.dim(' Score breakdown:'));
for (const reason of rec.reasons.filter(r => r.weight > 0)) {
console.log(` ${colors.muted(symbols.stepActive)} ${reason.description} (+${reason.weight})`);
}
if (qualityScore !== null && qualityScore !== undefined) {
const grade = getQualityGradeFromScore(qualityScore);
console.log(` ${colors.muted(symbols.stepActive)} Quality: ${qualityScore}/100 (${grade})`);
}
}
if (rec.warnings.length > 0) {
for (const warning of rec.warnings) {
console.log(` ${colors.warning(symbols.warning)} ${warning}`);
}
}
console.log('');
}
console.log(colors.muted('Install with: skillkit install <source>'));
console.log(colors.muted('More details: skillkit recommend --explain --verbose'));
}
private async handleHybridSearch(engine: RecommendationEngine, query: string): Promise<number> {
if (!this.quiet && !this.json) {
header(`Hybrid Search: "${query}"`);
}
const s = !this.quiet && !this.json ? spinner() : null;
s?.start('Initializing hybrid search...');
try {
await engine.initHybridSearch();
s?.message('Searching...');
const results = await engine.hybridSearch({
query,
limit: this.limit ? parseInt(this.limit, 10) : 10,
hybrid: true,
enableExpansion: this.expand,
enableReranking: this.rerank,
filters: {
minScore: this.minScore ? parseInt(this.minScore, 10) : undefined,
},
});
s?.stop(`Found ${results.length} results`);
if (this.json) {
console.log(JSON.stringify(results, null, 2));
return 0;
}
if (results.length === 0) {
warn(`No skills found matching "${query}"`);
return 0;
}
console.log('');
console.log(colors.bold(`Hybrid search results for "${query}" (${results.length} found):`));
if (this.expand && results[0]?.expandedTerms?.length) {
console.log(colors.muted(` Expanded: ${results[0].expandedTerms.join(', ')}`));
}
console.log('');
for (const result of results) {
let relevanceColor: (text: string) => string;
if (result.relevance >= 70) {
relevanceColor = colors.success;
} else if (result.relevance >= 50) {
relevanceColor = colors.warning;
} else {
relevanceColor = colors.muted;
}
const relevanceBar = progressBar(result.relevance, 100, 10);
console.log(` ${relevanceColor(`${result.relevance}%`)} ${colors.dim(relevanceBar)} ${colors.bold(result.skill.name)}`);
if (result.skill.description) {
console.log(` ${colors.muted(truncate(result.skill.description, 70))}`);
}
if (this.verbose) {
const scores: string[] = [];
if (typeof result.vectorSimilarity === 'number') {
scores.push(`vector: ${(result.vectorSimilarity * 100).toFixed(0)}%`);
}
if (typeof result.keywordScore === 'number') {
scores.push(`keyword: ${result.keywordScore.toFixed(0)}%`);
}
if (typeof result.rrfScore === 'number') {
scores.push(`rrf: ${result.rrfScore.toFixed(3)}`);
}
if (scores.length > 0) {
console.log(` ${colors.dim('Scores:')} ${scores.join(' | ')}`);
}
}
if (result.matchedTerms.length > 0) {
console.log(` ${colors.dim('Matched:')} ${result.matchedTerms.join(', ')}`);
}
console.log('');
}
return 0;
} catch (err) {
s?.stop(colors.error('Hybrid search failed'));
console.log(colors.muted(err instanceof Error ? err.message : String(err)));
console.log(colors.muted('Falling back to standard search...'));
return this.handleSearch(engine, query);
}
}
private async buildHybridIndex(): Promise<number> {
if (!this.quiet) {
header('Build Hybrid Search Index');
}
const index = this.loadIndex();
if (!index || index.skills.length === 0) {
warn('No skill index found. Run --update first.');
return 1;
}
const s = spinner();
s.start('Initializing...');
try {
const engine = new RecommendationEngine();
engine.loadIndex(index);
await engine.buildHybridIndex((progress) => {
const percentage = Math.round((progress.current / progress.total) * 100);
s.message(`${progress.phase}: ${progress.message || ''} (${percentage}%)`);
});
s.stop(colors.success(`${symbols.success} Built hybrid index for ${index.skills.length} skills`));
console.log(colors.muted(' Index stored in: ~/.skillkit/search.db'));
console.log(colors.muted(' Use --hybrid flag for vector+keyword search\n'));
return 0;
} catch (err) {
s.stop(colors.error('Failed to build hybrid index'));
console.log(colors.muted(err instanceof Error ? err.message : String(err)));
return 1;
}
}
private handleSearch(engine: RecommendationEngine, query: string): number {
if (!this.quiet && !this.json) {
header(`Search: "${query}"`);
+5 -6
View File
@@ -1,7 +1,7 @@
import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'node:fs';
import { dirname } from 'node:path';
import { Command, Option } from 'clipanion';
import { loadConfig, findAllSkills } from '@skillkit/core';
import { findAllSkills } from '@skillkit/core';
import type { AgentType } from '@skillkit/core';
import { getAdapter, detectAgent, getAllAdapters } from '@skillkit/agents';
import { getSearchDirs, getAgentConfigPath } from '../helpers.js';
@@ -67,10 +67,9 @@ export class SyncCommand extends Command {
if (this.agent) {
agentType = this.agent as AgentType;
} else if (isInteractive) {
// Let user select agent
// Let user select agent - always detect, don't rely on default config
s.start('Detecting agent...');
const config = loadConfig();
const detected = config.agent || (await detectAgent());
const detected = await detectAgent();
s.stop(`Detected: ${formatAgent(detected)}`);
const allAgents = getAllAdapters().map(a => a.type);
@@ -92,8 +91,8 @@ export class SyncCommand extends Command {
agentType = agentResult as AgentType;
} else {
const config = loadConfig();
agentType = config.agent || (await detectAgent());
// Non-interactive: always detect, don't rely on default config
agentType = await detectAgent();
}
const adapter = getAdapter(agentType);
+266
View File
@@ -0,0 +1,266 @@
import { Command, Option } from 'clipanion';
import { join } from 'node:path';
import { homedir } from 'node:os';
import {
loadIndex as loadIndexFromCache,
generateSkillTree,
saveTree,
loadTree,
type SkillTree,
type TreeNode,
} from '@skillkit/core';
import {
header,
colors,
symbols,
spinner,
warn,
} from '../onboarding/index.js';
const TREE_PATH = join(homedir(), '.skillkit', 'skill-tree.json');
export class TreeCommand extends Command {
static override paths = [['tree']];
static override usage = Command.Usage({
description: 'Browse skills in a hierarchical tree structure',
details: `
The tree command displays skills organized in a hierarchical taxonomy.
Navigate through categories like Development, Testing, DevOps, AI/ML, etc.
Features:
- Visual tree structure of all skills
- Filter by category path (e.g., "Frontend > React")
- Generate tree from skill index
- Export to markdown format
`,
examples: [
['Show full tree', '$0 tree'],
['Show specific category', '$0 tree Frontend'],
['Show subcategory', '$0 tree "Frontend > React"'],
['Limit depth', '$0 tree --depth 2'],
['Generate/update tree', '$0 tree --generate'],
['Export to markdown', '$0 tree --markdown'],
['Show tree stats', '$0 tree --stats'],
],
});
treePath = Option.String({ required: false });
depth = Option.String('--depth,-d', {
description: 'Maximum depth to display',
});
generate = Option.Boolean('--generate,-g', false, {
description: 'Generate/update tree from skill index',
});
markdown = Option.Boolean('--markdown,-m', false, {
description: 'Output in markdown format',
});
stats = Option.Boolean('--stats,-s', false, {
description: 'Show tree statistics',
});
json = Option.Boolean('--json,-j', false, {
description: 'Output in JSON format',
});
quiet = Option.Boolean('--quiet,-q', false, {
description: 'Minimal output',
});
async execute(): Promise<number> {
if (this.generate) {
return await this.generateTree();
}
const tree = this.loadOrGenerateTree();
if (!tree) {
warn('No skill tree found. Run "skillkit tree --generate" first.');
return 1;
}
if (this.stats) {
return this.showStats(tree);
}
if (this.json) {
console.log(JSON.stringify(tree, null, 2));
return 0;
}
if (this.markdown) {
const { treeToMarkdown } = await import('@skillkit/core');
console.log(treeToMarkdown(tree));
return 0;
}
return this.displayTree(tree);
}
private async generateTree(): Promise<number> {
if (!this.quiet) {
header('Generate Skill Tree');
}
const index = loadIndexFromCache();
if (!index || index.skills.length === 0) {
warn('No skill index found. Run "skillkit recommend --update" first.');
return 1;
}
const s = spinner();
s.start('Generating skill tree...');
try {
const tree = generateSkillTree(index.skills);
saveTree(tree, TREE_PATH);
s.stop(`Generated tree with ${tree.totalCategories} categories`);
console.log('');
console.log(colors.success(`${symbols.success} Tree generated successfully`));
console.log(colors.muted(` Total skills: ${tree.totalSkills}`));
console.log(colors.muted(` Categories: ${tree.totalCategories}`));
console.log(colors.muted(` Max depth: ${tree.maxDepth}`));
console.log(colors.muted(` Saved to: ${TREE_PATH}`));
console.log('');
return 0;
} catch (err) {
s.stop(colors.error('Failed to generate tree'));
console.log(colors.muted(err instanceof Error ? err.message : String(err)));
return 1;
}
}
private loadOrGenerateTree(): SkillTree | null {
let tree = loadTree(TREE_PATH);
if (!tree) {
const index = loadIndexFromCache();
if (index && index.skills.length > 0) {
tree = generateSkillTree(index.skills);
saveTree(tree, TREE_PATH);
}
}
return tree;
}
private showStats(tree: SkillTree): number {
if (!this.quiet) {
header('Skill Tree Statistics');
}
console.log('');
console.log(colors.bold('Overview:'));
console.log(` Total Skills: ${colors.accent(String(tree.totalSkills))}`);
console.log(` Categories: ${colors.accent(String(tree.totalCategories))}`);
console.log(` Max Depth: ${colors.accent(String(tree.maxDepth))}`);
console.log(` Generated: ${colors.muted(tree.generatedAt)}`);
console.log('');
console.log(colors.bold('Top-Level Categories:'));
for (const child of tree.rootNode.children) {
const percentage = tree.totalSkills > 0
? ((child.skillCount / tree.totalSkills) * 100).toFixed(1)
: '0.0';
console.log(
` ${colors.accent(child.name.padEnd(15))} ${String(child.skillCount).padStart(6)} skills (${percentage}%)`
);
if (child.children.length > 0) {
const subcats = child.children
.sort((a, b) => b.skillCount - a.skillCount)
.slice(0, 3)
.map((c) => `${c.name} (${c.skillCount})`)
.join(', ');
console.log(` ${colors.muted(subcats)}`);
}
}
console.log('');
return 0;
}
private displayTree(tree: SkillTree): number {
if (!this.quiet) {
header('Skill Tree');
console.log(colors.muted(`${tree.totalSkills} skills in ${tree.totalCategories} categories`));
console.log('');
}
let targetNode: TreeNode = tree.rootNode;
if (this.treePath) {
const segments = this.treePath.split('>').map((s) => s.trim());
let current = tree.rootNode;
for (const segment of segments) {
const child = current.children.find(
(c) => c.name.toLowerCase() === segment.toLowerCase()
);
if (!child) {
warn(`Category not found: ${segment}`);
console.log(colors.muted(`Available categories: ${current.children.map((c) => c.name).join(', ')}`));
return 1;
}
current = child;
}
targetNode = current;
}
let maxDepth = this.depth ? parseInt(this.depth, 10) : 3;
if (Number.isNaN(maxDepth) || maxDepth < 0) {
warn('Invalid depth value. Using default depth of 3.');
maxDepth = 3;
}
this.renderNode(targetNode, '', true, 0, maxDepth);
console.log('');
console.log(colors.muted('Navigate: skillkit tree "Category > Subcategory"'));
console.log(colors.muted('Generate: skillkit tree --generate'));
return 0;
}
private renderNode(
node: TreeNode,
prefix: string,
isLast: boolean,
depth: number,
maxDepth: number
): void {
if (depth > maxDepth) return;
const connector = depth === 0 ? '' : isLast ? '└── ' : '├── ';
const icon = node.children.length > 0 ? '📁 ' : '📄 ';
const skillInfo =
node.skillCount > 0 ? colors.muted(` (${node.skillCount})`) : '';
const nameColor = depth === 0 ? colors.bold : depth === 1 ? colors.accent : colors.dim;
console.log(`${prefix}${connector}${icon}${nameColor(node.name)}${skillInfo}`);
if (depth === maxDepth && node.skills.length > 0 && node.skills.length <= 5) {
const childPrefix = prefix + (isLast ? ' ' : '│ ');
for (const skill of node.skills) {
console.log(`${childPrefix} • ${colors.muted(skill)}`);
}
}
const newPrefix = prefix + (depth === 0 ? '' : isLast ? ' ' : '│ ');
if (depth < maxDepth) {
node.children.forEach((child, index) => {
const childIsLast = index === node.children.length - 1;
this.renderNode(child, newPrefix, childIsLast, depth + 1, maxDepth);
});
} else if (node.children.length > 0) {
console.log(`${newPrefix} ${colors.muted(`... ${node.children.length} more subcategories`)}`);
}
}
}
+11 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/core",
"version": "1.8.1",
"version": "1.10.0",
"description": "Core functionality for SkillKit - skill discovery, parsing, and translation",
"type": "module",
"main": "./dist/index.js",
@@ -37,6 +37,10 @@
"./learning": {
"import": "./dist/learning/index.js",
"types": "./dist/learning/index.d.ts"
},
"./search": {
"import": "./dist/search/index.js",
"types": "./dist/search/index.d.ts"
}
},
"files": [
@@ -53,7 +57,13 @@
"yaml": "^2.6.1",
"zod": "^3.24.1"
},
"optionalDependencies": {
"node-llama-cpp": "^3.15.0",
"better-sqlite3": "^12.0.0",
"sqlite-vec": "^0.1.6"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.11",
"@types/node": "^22.10.5",
"tsup": "^8.3.5",
"typescript": "^5.7.2",
@@ -0,0 +1,332 @@
/**
* Skill to Subagent Converter Tests
*/
import { describe, it, expect } from 'vitest';
import {
skillToSubagent,
generateSubagentFromSkill,
} from '../skill-converter.js';
import type { Skill } from '../../types.js';
describe('Skill Converter', () => {
const mockSkill: Skill = {
name: 'code-simplifier',
description: 'Simplifies and refines code for clarity and maintainability',
path: '/path/to/skills/code-simplifier',
location: 'project',
enabled: true,
};
const mockSkillContent = `---
name: code-simplifier
description: Simplifies and refines code for clarity and maintainability
version: "1.0.0"
author: skillkit
tags: [refactoring, code-quality]
allowed-tools: Edit, Read, Grep
---
# Code Simplifier
You are an expert at simplifying code.
## Guidelines
- Remove unnecessary complexity
- Improve readability
- Preserve functionality
`;
describe('skillToSubagent', () => {
it('should convert skill to canonical agent in reference mode', () => {
const result = skillToSubagent(mockSkill, mockSkillContent);
expect(result.name).toBe('code-simplifier');
expect(result.description).toBe('Simplifies and refines code for clarity and maintainability');
expect(result.skills).toEqual(['code-simplifier']);
expect(result.sourceFormat).toBe('claude-agent');
expect(result.userInvocable).toBe(true);
});
it('should not include skills array in inline mode', () => {
const result = skillToSubagent(mockSkill, mockSkillContent, { inline: true });
expect(result.name).toBe('code-simplifier');
expect(result.skills).toBeUndefined();
expect(result.content).toContain('Code Simplifier');
expect(result.content).toContain('Remove unnecessary complexity');
});
it('should parse allowed-tools from skill frontmatter', () => {
const result = skillToSubagent(mockSkill, mockSkillContent);
expect(result.allowedTools).toEqual(['Edit', 'Read', 'Grep']);
});
it('should use option tools over frontmatter tools', () => {
const result = skillToSubagent(mockSkill, mockSkillContent, {
tools: ['Bash', 'Write'],
});
expect(result.allowedTools).toEqual(['Bash', 'Write']);
});
it('should set model from options', () => {
const result = skillToSubagent(mockSkill, mockSkillContent, {
model: 'opus',
});
expect(result.model).toBe('opus');
});
it('should not set model when inherit is specified', () => {
const result = skillToSubagent(mockSkill, mockSkillContent, {
model: 'inherit',
});
expect(result.model).toBeUndefined();
});
it('should set permission mode from options', () => {
const result = skillToSubagent(mockSkill, mockSkillContent, {
permissionMode: 'plan',
});
expect(result.permissionMode).toBe('plan');
});
it('should set disallowed tools from options', () => {
const result = skillToSubagent(mockSkill, mockSkillContent, {
disallowedTools: ['Bash', 'Write'],
});
expect(result.disallowedTools).toEqual(['Bash', 'Write']);
});
it('should preserve version and author from frontmatter', () => {
const result = skillToSubagent(mockSkill, mockSkillContent);
expect(result.version).toBe('1.0.0');
expect(result.author).toBe('skillkit');
});
it('should preserve tags from frontmatter', () => {
const result = skillToSubagent(mockSkill, mockSkillContent);
expect(result.tags).toEqual(['refactoring', 'code-quality']);
});
});
describe('generateSubagentFromSkill', () => {
it('should generate valid markdown with frontmatter in reference mode', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent);
expect(content).toContain('---');
expect(content).toContain('name: code-simplifier');
expect(content).toContain('description: Simplifies and refines code');
expect(content).toContain('skills:');
expect(content).toContain(' - code-simplifier');
expect(content).toContain('user-invocable: true');
});
it('should generate valid markdown in inline mode', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent, { inline: true });
expect(content).toContain('---');
expect(content).toContain('name: code-simplifier');
expect(content).not.toContain('skills:');
expect(content).toContain('# Code Simplifier');
expect(content).toContain('Remove unnecessary complexity');
});
it('should include tools in frontmatter', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent);
expect(content).toContain('tools:');
expect(content).toContain(' - Edit');
expect(content).toContain(' - Read');
expect(content).toContain(' - Grep');
});
it('should include model in frontmatter when specified', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent, {
model: 'opus',
});
expect(content).toContain('model: opus');
});
it('should include permission mode in frontmatter when specified', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent, {
permissionMode: 'plan',
});
expect(content).toContain('permissionMode: plan');
});
it('should include disallowed tools in frontmatter when specified', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent, {
disallowedTools: ['Bash'],
});
expect(content).toContain('disallowedTools:');
expect(content).toContain(' - Bash');
});
it('should include version and author in frontmatter', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent);
expect(content).toContain('version: "1.0.0"');
expect(content).toContain('author: skillkit');
});
it('should include tags in frontmatter', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent);
expect(content).toContain('tags: [refactoring, code-quality]');
});
it('should generate reference content with skill description', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent);
expect(content).toContain('# Code Simplifier');
expect(content).toContain('uses the "code-simplifier" skill');
expect(content).toContain('## Skill Description');
expect(content).toContain('Simplifies and refines code');
});
it('should embed full skill content in inline mode', () => {
const content = generateSubagentFromSkill(mockSkill, mockSkillContent, { inline: true });
expect(content).toContain('You are a specialized assistant');
expect(content).toContain('Remove unnecessary complexity');
expect(content).toContain('Improve readability');
expect(content).toContain('Preserve functionality');
});
});
describe('edge cases', () => {
it('should handle skill without frontmatter', () => {
const noFrontmatterContent = `# Simple Skill
Just some content without frontmatter.
`;
const result = skillToSubagent(mockSkill, noFrontmatterContent);
expect(result.name).toBe('code-simplifier');
expect(result.description).toBe('Simplifies and refines code for clarity and maintainability');
expect(result.allowedTools).toBeUndefined();
});
it('should handle skill with empty allowed-tools', () => {
const contentWithEmptyTools = `---
name: test-skill
description: Test
allowed-tools:
---
Content here.
`;
const skill: Skill = {
name: 'test-skill',
description: 'Test',
path: '/path/to/test-skill',
location: 'project',
enabled: true,
};
const result = skillToSubagent(skill, contentWithEmptyTools);
expect(result.allowedTools).toBeUndefined();
});
it('should handle skill name with multiple hyphens', () => {
const skill: Skill = {
name: 'my-complex-skill-name',
description: 'A complex skill',
path: '/path/to/skill',
location: 'project',
enabled: true,
};
const content = generateSubagentFromSkill(skill, `---
name: my-complex-skill-name
description: A complex skill
---
Content.
`);
expect(content).toContain('# My Complex Skill Name');
});
it('should handle space-separated allowed-tools format', () => {
const contentWithSpaceSeparated = `---
name: test-skill
description: Test
allowed-tools: Bash(git:*) Read Write Edit
---
Content here.
`;
const skill: Skill = {
name: 'test-skill',
description: 'Test',
path: '/path/to/test-skill',
location: 'project',
enabled: true,
};
const result = skillToSubagent(skill, contentWithSpaceSeparated);
expect(result.allowedTools).toEqual(['Bash(git:*)', 'Read', 'Write', 'Edit']);
});
it('should handle YAML array format for allowed-tools', () => {
const contentWithArrayTools = `---
name: test-skill
description: Test
allowed-tools:
- Read
- Write
- Bash(npm:*)
---
Content here.
`;
const skill: Skill = {
name: 'test-skill',
description: 'Test',
path: '/path/to/test-skill',
location: 'project',
enabled: true,
};
const result = skillToSubagent(skill, contentWithArrayTools);
expect(result.allowedTools).toEqual(['Read', 'Write', 'Bash(npm:*)']);
});
it('should handle CRLF line endings', () => {
const crlfContent = `---\r\nname: test-skill\r\ndescription: Test\r\n---\r\n\r\n# Content\r\n\r\nWith CRLF endings.\r\n`;
const skill: Skill = {
name: 'test-skill',
description: 'Test',
path: '/path/to/test-skill',
location: 'project',
enabled: true,
};
const result = skillToSubagent(skill, crlfContent, { inline: true });
expect(result.content).toContain('# Content');
expect(result.content).toContain('With CRLF endings.');
});
});
});
+8
View File
@@ -60,3 +60,11 @@ export {
getAgentTargetDirectory,
isAgentCompatible,
} from './translator.js';
// Skill Converter
export {
skillToSubagent,
generateSubagentFromSkill,
loadAndConvertSkill,
type SkillToSubagentOptions,
} from './skill-converter.js';
+285
View File
@@ -0,0 +1,285 @@
/**
* Skill to Subagent Converter
*
* Converts SkillKit skills into Claude Code native subagent format (.md files).
* Supports both reference mode (skills: [skill-name]) and inline mode (embedded content).
*/
import type { Skill, SkillFrontmatter } from '../types.js';
import type { CanonicalAgent, AgentPermissionMode } from './types.js';
import { extractFrontmatter, readSkillContent } from '../skills.js';
/**
* Options for skill-to-subagent conversion
*/
export interface SkillToSubagentOptions {
/** Embed full skill content in system prompt (default: false - use skills reference) */
inline?: boolean;
/** Model to use (sonnet, opus, haiku, inherit) */
model?: 'sonnet' | 'opus' | 'haiku' | 'inherit';
/** Permission mode for the agent */
permissionMode?: AgentPermissionMode;
/** Tools the agent is allowed to use */
tools?: string[];
/** Tools the agent cannot use */
disallowedTools?: string[];
}
/**
* Convert a skill to a CanonicalAgent representation
*/
export function skillToSubagent(
skill: Skill,
skillContent: string,
options?: SkillToSubagentOptions
): CanonicalAgent {
const frontmatter = extractFrontmatter(skillContent) as Partial<SkillFrontmatter> | null;
const inline = options?.inline ?? false;
const allowedTools = parseAllowedTools(frontmatter, options?.tools);
const content = inline
? generateInlineContent(skill, skillContent)
: generateReferenceContent(skill, frontmatter);
return {
name: skill.name,
description: skill.description,
model: options?.model === 'inherit' ? undefined : options?.model,
permissionMode: options?.permissionMode,
disallowedTools: options?.disallowedTools,
allowedTools,
skills: inline ? undefined : [skill.name],
content,
sourceFormat: 'claude-agent',
sourceAgent: 'claude-code',
version: frontmatter?.version,
author: frontmatter?.author,
tags: frontmatter?.tags,
userInvocable: true,
};
}
/**
* Generate the markdown content for a Claude Code subagent from a skill
*/
export function generateSubagentFromSkill(
skill: Skill,
skillContent: string,
options?: SkillToSubagentOptions
): string {
const canonical = skillToSubagent(skill, skillContent, options);
return generateSubagentMarkdown(canonical, options?.inline ?? false);
}
/**
* Parse allowed-tools from skill frontmatter
* Handles both string (comma or space-separated) and array formats
*/
function parseAllowedTools(
frontmatter: Partial<SkillFrontmatter> | null,
optionTools?: string[]
): string[] | undefined {
if (optionTools && optionTools.length > 0) {
return optionTools;
}
const allowedTools = frontmatter?.['allowed-tools'];
if (!allowedTools) {
return undefined;
}
if (Array.isArray(allowedTools)) {
const result = allowedTools
.map(t => (typeof t === 'string' ? t.trim() : ''))
.filter(Boolean);
return result.length > 0 ? result : undefined;
}
if (typeof allowedTools === 'string') {
const result = allowedTools
.split(/[,\s]+/)
.map(t => t.trim())
.filter(Boolean);
return result.length > 0 ? result : undefined;
}
return undefined;
}
/**
* Generate inline content (full skill embedded)
*/
function generateInlineContent(skill: Skill, skillContent: string): string {
const bodyContent = extractBodyContent(skillContent);
const lines: string[] = [];
lines.push(`# ${formatAgentName(skill.name)}`);
lines.push('');
lines.push('You are a specialized assistant powered by the following skill.');
lines.push('');
lines.push('---');
lines.push('');
lines.push(bodyContent.trim());
return lines.join('\n');
}
/**
* Generate reference content (skill referenced by name)
*/
function generateReferenceContent(
skill: Skill,
frontmatter: Partial<SkillFrontmatter> | null
): string {
const lines: string[] = [];
lines.push(`# ${formatAgentName(skill.name)}`);
lines.push('');
lines.push(`You are a specialized assistant that uses the "${skill.name}" skill.`);
lines.push('');
lines.push('## Skill Description');
lines.push('');
lines.push(skill.description);
lines.push('');
lines.push('## Usage');
lines.push('');
lines.push(`This agent automatically loads the "${skill.name}" skill.`);
lines.push('Follow the skill instructions to complete tasks effectively.');
if (frontmatter?.tags && frontmatter.tags.length > 0) {
lines.push('');
lines.push('## Related Topics');
lines.push('');
lines.push(frontmatter.tags.map(t => `- ${t}`).join('\n'));
}
return lines.join('\n');
}
/**
* Generate full subagent markdown file content
*/
function generateSubagentMarkdown(
canonical: CanonicalAgent,
isInline: boolean
): string {
const lines: string[] = ['---'];
lines.push(`name: ${escapeYamlString(canonical.name)}`);
lines.push(`description: ${escapeYamlString(canonical.description)}`);
if (canonical.model) {
lines.push(`model: ${canonical.model}`);
}
if (canonical.permissionMode) {
lines.push(`permissionMode: ${canonical.permissionMode}`);
}
appendYamlList(lines, 'tools', canonical.allowedTools);
appendYamlList(lines, 'disallowedTools', canonical.disallowedTools);
if (!isInline) {
appendYamlList(lines, 'skills', canonical.skills);
}
if (canonical.version) {
lines.push(`version: "${canonical.version}"`);
}
if (canonical.author) {
lines.push(`author: ${escapeYamlString(canonical.author)}`);
}
if (canonical.tags && canonical.tags.length > 0) {
lines.push(`tags: [${canonical.tags.map(t => escapeYamlString(t)).join(', ')}]`);
}
if (canonical.userInvocable !== undefined) {
lines.push(`user-invocable: ${canonical.userInvocable}`);
}
lines.push('---', '', canonical.content);
return lines.join('\n');
}
function appendYamlList(lines: string[], key: string, items?: string[]): void {
if (!items || items.length === 0) return;
lines.push(`${key}:`);
for (const item of items) {
lines.push(` - ${escapeYamlString(item)}`);
}
}
/**
* Extract body content from skill markdown (without frontmatter)
* Handles both Unix (LF) and Windows (CRLF) line endings
*/
function extractBodyContent(content: string): string {
const normalized = content.replace(/\r\n/g, '\n');
const match = normalized.match(/^---\s*\n[\s\S]*?\n---\s*\n([\s\S]*)$/);
if (match) {
return match[1];
}
return content;
}
/**
* Format agent name for display (kebab-case to Title Case)
*/
function formatAgentName(name: string): string {
return name
.split('-')
.map(word => word.charAt(0).toUpperCase() + word.slice(1))
.join(' ');
}
/**
* Escape special characters in YAML strings
* Only escapes when necessary for valid YAML parsing
*/
function escapeYamlString(str: string): string {
if (
str.includes('\n') ||
str.includes(':') ||
str.includes('#') ||
str.startsWith('-') ||
str.startsWith('*') ||
str.startsWith('&') ||
str.startsWith('!') ||
str.startsWith('{') ||
str.startsWith('[') ||
str.startsWith('>') ||
str.startsWith('|') ||
str.startsWith('@') ||
str.startsWith('`') ||
/^['"]/.test(str)
) {
return `"${str.replace(/"/g, '\\"').replace(/\n/g, '\\n')}"`;
}
return str;
}
/**
* Load a skill and generate subagent content
* Convenience function that reads skill content and generates subagent markdown
*/
export function loadAndConvertSkill(
skill: Skill,
options?: SkillToSubagentOptions
): { success: boolean; content?: string; error?: string } {
const skillContent = readSkillContent(skill.path);
if (!skillContent) {
return {
success: false,
error: `Could not read skill content from: ${skill.path}`,
};
}
try {
const content = generateSubagentFromSkill(skill, skillContent, options);
return { success: true, content };
} catch (err) {
return {
success: false,
error: err instanceof Error ? err.message : 'Unknown error',
};
}
}
+6 -1
View File
@@ -64,7 +64,12 @@ export const AgentFrontmatter = z.object({
/** Allowed tools for this agent */
allowedTools: z.union([
z.array(z.string()),
z.string(), // YAML-style list in frontmatter
z.string(),
]).optional(),
/** Tools (Claude Code native format alias for allowedTools) */
tools: z.union([
z.array(z.string()),
z.string(),
]).optional(),
/** Agent-scoped hooks */
hooks: z.array(AgentHook).optional(),
+22
View File
@@ -0,0 +1,22 @@
export * from './types.js';
export * from './utils.js';
export {
STANDARD_PLACEHOLDERS,
ConnectorCategorySchema,
ConnectorPlaceholderSchema,
ConnectorMappingSchema,
ConnectorConfigSchema,
} from './types.js';
export {
detectPlaceholders,
analyzePlaceholders,
replacePlaceholders,
applyConnectorConfig,
getPlaceholderInfo,
generateConnectorsMarkdown,
createConnectorConfig,
validateConnectorConfig,
suggestMappingsFromMcp,
} from './utils.js';
+162
View File
@@ -0,0 +1,162 @@
import { z } from 'zod';
export const ConnectorCategorySchema = z.enum([
'crm',
'chat',
'email',
'calendar',
'docs',
'data',
'search',
'enrichment',
'analytics',
'storage',
'notifications',
'ai',
'custom',
]);
export type ConnectorCategory = z.infer<typeof ConnectorCategorySchema>;
export const ConnectorPlaceholderSchema = z.object({
placeholder: z.string(),
category: ConnectorCategorySchema,
description: z.string(),
examples: z.array(z.string()).default([]),
required: z.boolean().default(false),
});
export type ConnectorPlaceholder = z.infer<typeof ConnectorPlaceholderSchema>;
export const ConnectorMappingSchema = z.object({
placeholder: z.string(),
tool: z.string(),
mcpServer: z.string().optional(),
config: z.record(z.unknown()).optional(),
});
export type ConnectorMapping = z.infer<typeof ConnectorMappingSchema>;
export const ConnectorConfigSchema = z.object({
version: z.number().default(1),
mappings: z.array(ConnectorMappingSchema),
description: z.string().optional(),
});
export type ConnectorConfig = z.infer<typeof ConnectorConfigSchema>;
export const STANDARD_PLACEHOLDERS: Record<ConnectorCategory, ConnectorPlaceholder> = {
crm: {
placeholder: '~~CRM',
category: 'crm',
description: 'Customer Relationship Management system',
examples: ['Salesforce', 'HubSpot', 'Pipedrive', 'Zoho CRM'],
required: false,
},
chat: {
placeholder: '~~chat',
category: 'chat',
description: 'Team communication and messaging platform',
examples: ['Slack', 'Microsoft Teams', 'Discord'],
required: false,
},
email: {
placeholder: '~~email',
category: 'email',
description: 'Email service for sending and receiving messages',
examples: ['Gmail', 'Outlook', 'SendGrid'],
required: false,
},
calendar: {
placeholder: '~~calendar',
category: 'calendar',
description: 'Calendar and scheduling system',
examples: ['Google Calendar', 'Outlook Calendar', 'Calendly'],
required: false,
},
docs: {
placeholder: '~~docs',
category: 'docs',
description: 'Document management and collaboration',
examples: ['Google Docs', 'Notion', 'Confluence', 'Coda'],
required: false,
},
data: {
placeholder: '~~data',
category: 'data',
description: 'Data warehouse or database',
examples: ['BigQuery', 'Snowflake', 'PostgreSQL', 'Databricks'],
required: false,
},
search: {
placeholder: '~~search',
category: 'search',
description: 'Enterprise search across tools',
examples: ['Glean', 'Elastic', 'Algolia'],
required: false,
},
enrichment: {
placeholder: '~~enrichment',
category: 'enrichment',
description: 'Data enrichment service',
examples: ['Clearbit', 'ZoomInfo', 'Apollo'],
required: false,
},
analytics: {
placeholder: '~~analytics',
category: 'analytics',
description: 'Analytics and reporting platform',
examples: ['Mixpanel', 'Amplitude', 'Google Analytics'],
required: false,
},
storage: {
placeholder: '~~storage',
category: 'storage',
description: 'File storage service',
examples: ['Google Drive', 'Dropbox', 'S3', 'Box'],
required: false,
},
notifications: {
placeholder: '~~notifications',
category: 'notifications',
description: 'Notification and alerting service',
examples: ['PagerDuty', 'Opsgenie', 'SMS'],
required: false,
},
ai: {
placeholder: '~~ai',
category: 'ai',
description: 'AI/ML model or service',
examples: ['OpenAI', 'Anthropic', 'Hugging Face'],
required: false,
},
custom: {
placeholder: '~~custom',
category: 'custom',
description: 'Custom integration',
examples: [],
required: false,
},
};
export interface PlaceholderMatch {
placeholder: string;
category: ConnectorCategory | null;
line: number;
column: number;
context: string;
}
export interface PlaceholderReplacement {
original: string;
replacement: string;
category: ConnectorCategory | null;
}
export interface ConnectorAnalysis {
placeholders: PlaceholderMatch[];
categories: ConnectorCategory[];
requiredCount: number;
optionalCount: number;
hasUnknownPlaceholders: boolean;
}
+268
View File
@@ -0,0 +1,268 @@
import {
type ConnectorCategory,
type ConnectorPlaceholder,
type ConnectorMapping,
type ConnectorConfig,
type PlaceholderMatch,
type PlaceholderReplacement,
type ConnectorAnalysis,
STANDARD_PLACEHOLDERS,
ConnectorCategorySchema,
} from './types.js';
const PLACEHOLDER_REGEX = /~~(\w+)/g;
export function detectPlaceholders(content: string): PlaceholderMatch[] {
const matches: PlaceholderMatch[] = [];
const lines = content.split('\n');
for (let lineNum = 0; lineNum < lines.length; lineNum++) {
const line = lines[lineNum];
let match: RegExpExecArray | null;
const regex = new RegExp(PLACEHOLDER_REGEX.source, 'g');
while ((match = regex.exec(line)) !== null) {
const placeholder = match[0];
const categoryName = match[1].toLowerCase();
let category: ConnectorCategory | null = null;
try {
category = ConnectorCategorySchema.parse(categoryName);
} catch {
category = null;
}
const contextStart = Math.max(0, match.index - 20);
const contextEnd = Math.min(line.length, match.index + placeholder.length + 20);
const context = line.slice(contextStart, contextEnd);
matches.push({
placeholder,
category,
line: lineNum + 1,
column: match.index + 1,
context: context.trim(),
});
}
}
return matches;
}
export function analyzePlaceholders(content: string): ConnectorAnalysis {
const matches = detectPlaceholders(content);
const categories = new Set<ConnectorCategory>();
let requiredCount = 0;
let optionalCount = 0;
let hasUnknownPlaceholders = false;
for (const match of matches) {
if (match.category) {
categories.add(match.category);
const standardPlaceholder = STANDARD_PLACEHOLDERS[match.category];
if (standardPlaceholder?.required) {
requiredCount++;
} else {
optionalCount++;
}
} else {
hasUnknownPlaceholders = true;
}
}
return {
placeholders: matches,
categories: Array.from(categories),
requiredCount,
optionalCount,
hasUnknownPlaceholders,
};
}
export function replacePlaceholders(
content: string,
mappings: ConnectorMapping[]
): { result: string; replacements: PlaceholderReplacement[] } {
const replacements: PlaceholderReplacement[] = [];
let result = content;
for (const mapping of mappings) {
const regex = new RegExp(escapeRegExp(mapping.placeholder), 'g');
const originalMatches = content.match(regex);
if (originalMatches) {
result = result.replace(regex, mapping.tool);
const categoryName = mapping.placeholder.replace('~~', '').toLowerCase();
let category: ConnectorCategory | null = null;
try {
category = ConnectorCategorySchema.parse(categoryName);
} catch {
category = null;
}
replacements.push({
original: mapping.placeholder,
replacement: mapping.tool,
category,
});
}
}
return { result, replacements };
}
export function applyConnectorConfig(
content: string,
config: ConnectorConfig
): { result: string; replacements: PlaceholderReplacement[] } {
return replacePlaceholders(content, config.mappings);
}
export function getPlaceholderInfo(placeholder: string): ConnectorPlaceholder | null {
const categoryName = placeholder.replace('~~', '').toLowerCase();
try {
const category = ConnectorCategorySchema.parse(categoryName);
return STANDARD_PLACEHOLDERS[category];
} catch {
return null;
}
}
export function generateConnectorsMarkdown(analysis: ConnectorAnalysis): string {
const lines: string[] = [];
lines.push('# Connectors');
lines.push('');
lines.push('This skill uses the following connector placeholders. Replace them with your specific tools.');
lines.push('');
lines.push('| Placeholder | Category | Description | Examples |');
lines.push('|-------------|----------|-------------|----------|');
for (const match of analysis.placeholders) {
if (match.category) {
const info = STANDARD_PLACEHOLDERS[match.category];
lines.push(
`| \`${match.placeholder}\` | ${match.category} | ${info.description} | ${info.examples.slice(0, 3).join(', ')} |`
);
} else {
lines.push(
`| \`${match.placeholder}\` | custom | Custom integration | - |`
);
}
}
lines.push('');
lines.push('## How to Customize');
lines.push('');
lines.push('1. Find all placeholders: `grep -n "~~" SKILL.md`');
lines.push('2. Replace each `~~category` with your specific tool name');
lines.push('3. Configure MCP server in `.mcp.json` if needed');
lines.push('');
return lines.join('\n');
}
export function createConnectorConfig(
mappings: Array<{ placeholder: string; tool: string; mcpServer?: string }>
): ConnectorConfig {
return {
version: 1,
mappings: mappings.map((m) => ({
placeholder: m.placeholder,
tool: m.tool,
mcpServer: m.mcpServer,
})),
};
}
export function validateConnectorConfig(
config: ConnectorConfig,
content: string
): { valid: boolean; errors: string[]; warnings: string[] } {
const errors: string[] = [];
const warnings: string[] = [];
const analysis = analyzePlaceholders(content);
const placeholdersInContent = new Set(analysis.placeholders.map((p) => p.placeholder));
const mappedPlaceholders = new Set(config.mappings.map((m) => m.placeholder));
for (const placeholder of placeholdersInContent) {
if (!mappedPlaceholders.has(placeholder)) {
const info = getPlaceholderInfo(placeholder);
if (info?.required) {
errors.push(`Required placeholder ${placeholder} is not mapped`);
} else {
warnings.push(`Optional placeholder ${placeholder} is not mapped`);
}
}
}
for (const mapping of config.mappings) {
if (!placeholdersInContent.has(mapping.placeholder)) {
warnings.push(`Mapping for ${mapping.placeholder} exists but placeholder not found in content`);
}
}
return {
valid: errors.length === 0,
errors,
warnings,
};
}
export function suggestMappingsFromMcp(
mcpServers: string[]
): ConnectorMapping[] {
const suggestions: ConnectorMapping[] = [];
const serverLower = mcpServers.map((s) => s.toLowerCase());
const serverToCategory: Record<string, { category: ConnectorCategory; tool: string }[]> = {
slack: [{ category: 'chat', tool: 'Slack' }],
teams: [{ category: 'chat', tool: 'Microsoft Teams' }],
discord: [{ category: 'chat', tool: 'Discord' }],
gmail: [{ category: 'email', tool: 'Gmail' }],
outlook: [{ category: 'email', tool: 'Outlook' }],
sendgrid: [{ category: 'email', tool: 'SendGrid' }],
salesforce: [{ category: 'crm', tool: 'Salesforce' }],
hubspot: [{ category: 'crm', tool: 'HubSpot' }],
pipedrive: [{ category: 'crm', tool: 'Pipedrive' }],
notion: [{ category: 'docs', tool: 'Notion' }],
confluence: [{ category: 'docs', tool: 'Confluence' }],
google: [
{ category: 'docs', tool: 'Google Docs' },
{ category: 'calendar', tool: 'Google Calendar' },
{ category: 'storage', tool: 'Google Drive' },
],
bigquery: [{ category: 'data', tool: 'BigQuery' }],
snowflake: [{ category: 'data', tool: 'Snowflake' }],
postgres: [{ category: 'data', tool: 'PostgreSQL' }],
supabase: [{ category: 'data', tool: 'Supabase' }],
clearbit: [{ category: 'enrichment', tool: 'Clearbit' }],
zoominfo: [{ category: 'enrichment', tool: 'ZoomInfo' }],
apollo: [{ category: 'enrichment', tool: 'Apollo' }],
openai: [{ category: 'ai', tool: 'OpenAI' }],
anthropic: [{ category: 'ai', tool: 'Anthropic Claude' }],
};
for (const server of serverLower) {
for (const [keyword, mappings] of Object.entries(serverToCategory)) {
if (server.includes(keyword)) {
for (const mapping of mappings) {
suggestions.push({
placeholder: STANDARD_PLACEHOLDERS[mapping.category].placeholder,
tool: mapping.tool,
mcpServer: server,
});
}
}
}
}
return suggestions;
}
function escapeRegExp(string: string): string {
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
+23
View File
@@ -0,0 +1,23 @@
export * from './types.js';
export * from './manager.js';
export * from './mode.js';
export {
ExecutionManager,
createExecutionManager,
} from './manager.js';
export {
ExecutionStepStatusSchema,
ExecutionStepSchema,
ExecutionFlowSchema,
} from './types.js';
export {
detectExecutionMode,
getModeDescription,
requireEnhancedMode,
requireCapability,
getStandaloneAlternative,
createModeAwareExecutor,
} from './mode.js';
+314
View File
@@ -0,0 +1,314 @@
import { randomUUID } from 'node:crypto';
import {
type ExecutionFlow,
type ExecutionStep,
type ExecutionFlowConfig,
type StepDefinition,
type ExecutionContext,
type FlowSummary,
type FlowMetrics,
ExecutionFlowSchema,
} from './types.js';
const DEFAULT_MAX_RETRIES = 3;
const DEFAULT_RETRY_DELAY = 1000;
const DEFAULT_TIMEOUT = 30000;
export class ExecutionManager {
private flows: Map<string, ExecutionFlow> = new Map();
private stepDefinitions: Map<string, Map<string, StepDefinition>> = new Map();
private config: ExecutionFlowConfig;
private metrics: FlowMetrics;
constructor(config: ExecutionFlowConfig = {}) {
this.config = config;
this.metrics = {
totalFlows: 0,
completedFlows: 0,
failedFlows: 0,
averageDuration: 0,
stepMetrics: new Map(),
};
}
createFlow(
skillName: string,
steps: Array<{ name: string; description?: string }>,
options: {
mode?: 'standalone' | 'enhanced';
mcpServers?: string[];
context?: Record<string, unknown>;
} = {}
): ExecutionFlow {
const flowId = randomUUID();
const executionSteps: ExecutionStep[] = steps.map((step, index) => ({
id: `${flowId}-step-${index}`,
name: step.name,
description: step.description,
status: 'pending',
retryCount: 0,
}));
const flow: ExecutionFlow = ExecutionFlowSchema.parse({
id: flowId,
skillName,
version: 1,
steps: executionSteps,
currentStepIndex: -1,
status: 'pending',
context: options.context,
mode: options.mode || 'standalone',
mcpServers: options.mcpServers,
});
this.flows.set(flowId, flow);
this.metrics.totalFlows++;
return flow;
}
registerStepDefinitions(skillName: string, definitions: StepDefinition[]): void {
const skillSteps = new Map<string, StepDefinition>();
for (const def of definitions) {
skillSteps.set(def.name, def);
}
this.stepDefinitions.set(skillName, skillSteps);
}
async executeFlow(flowId: string): Promise<ExecutionFlow> {
const flow = this.flows.get(flowId);
if (!flow) {
throw new Error(`Flow ${flowId} not found`);
}
flow.status = 'running';
flow.startedAt = new Date().toISOString();
const skillSteps = this.stepDefinitions.get(flow.skillName);
try {
let previousOutput: Record<string, unknown> = {};
for (let i = 0; i < flow.steps.length; i++) {
const step = flow.steps[i];
flow.currentStepIndex = i;
const context: ExecutionContext = {
flow,
stepIndex: i,
previousOutput,
mcpTools: flow.mcpServers,
isEnhanced: flow.mode === 'enhanced',
};
const stepDef = skillSteps?.get(step.name);
if (stepDef?.condition && !stepDef.condition(context)) {
step.status = 'skipped';
continue;
}
const result = await this.executeStep(step, stepDef, context);
if (result.status === 'completed' && result.output) {
previousOutput = { ...previousOutput, ...result.output };
}
if (result.status === 'failed') {
flow.status = 'failed';
flow.completedAt = new Date().toISOString();
flow.totalDuration = this.calculateDuration(flow.startedAt, flow.completedAt);
this.metrics.failedFlows++;
this.config.onFlowComplete?.(flow);
return flow;
}
}
flow.status = 'completed';
flow.completedAt = new Date().toISOString();
flow.totalDuration = this.calculateDuration(flow.startedAt, flow.completedAt);
this.metrics.completedFlows++;
this.updateAverageDuration(flow.totalDuration);
this.config.onFlowComplete?.(flow);
return flow;
} catch (error) {
flow.status = 'failed';
flow.completedAt = new Date().toISOString();
flow.totalDuration = this.calculateDuration(flow.startedAt, flow.completedAt);
this.metrics.failedFlows++;
throw error;
}
}
private async executeStep(
step: ExecutionStep,
stepDef: StepDefinition | undefined,
context: ExecutionContext
): Promise<ExecutionStep> {
const maxRetries = this.config.maxRetries ?? DEFAULT_MAX_RETRIES;
const retryDelay = this.config.retryDelay ?? DEFAULT_RETRY_DELAY;
const timeout = stepDef?.timeout ?? this.config.timeout ?? DEFAULT_TIMEOUT;
step.status = 'running';
step.startedAt = new Date().toISOString();
this.config.onStepStart?.(step, context.flow);
let lastError: Error | null = null;
while (step.retryCount <= maxRetries) {
let timeoutId: ReturnType<typeof setTimeout> | undefined;
try {
if (stepDef) {
const timeoutPromise = new Promise<never>((_, reject) => {
timeoutId = setTimeout(
() => reject(new Error(`Step ${step.name} timed out`)),
timeout
);
});
const executePromise = stepDef.execute(step.input || {}, context);
try {
step.output = await Promise.race([executePromise, timeoutPromise]);
} finally {
if (timeoutId) clearTimeout(timeoutId);
}
}
step.status = 'completed';
step.completedAt = new Date().toISOString();
step.duration = this.calculateDuration(step.startedAt, step.completedAt);
this.config.onStepComplete?.(step, context.flow);
this.updateStepMetrics(step.name, true, step.duration);
return step;
} catch (error) {
lastError = error instanceof Error ? error : new Error(String(error));
step.retryCount++;
if (stepDef?.retryable !== false && step.retryCount <= maxRetries) {
await this.delay(retryDelay * step.retryCount);
} else {
break;
}
}
}
step.status = 'failed';
step.error = lastError?.message || 'Unknown error';
step.completedAt = new Date().toISOString();
step.duration = this.calculateDuration(step.startedAt, step.completedAt);
this.config.onStepError?.(step, lastError || new Error('Unknown error'), context.flow);
this.updateStepMetrics(step.name, false, step.duration);
return step;
}
getFlow(flowId: string): ExecutionFlow | undefined {
return this.flows.get(flowId);
}
getFlowSummary(flowId: string): FlowSummary | null {
const flow = this.flows.get(flowId);
if (!flow) return null;
const completedSteps = flow.steps.filter(
(s) => s.status === 'completed' || s.status === 'skipped'
).length;
const currentStep =
flow.currentStepIndex >= 0 ? flow.steps[flow.currentStepIndex]?.name : undefined;
return {
id: flow.id,
skillName: flow.skillName,
status: flow.status,
progress: flow.steps.length > 0 ? (completedSteps / flow.steps.length) * 100 : 0,
currentStep,
startedAt: flow.startedAt,
completedAt: flow.completedAt,
totalDuration: flow.totalDuration,
stepsCompleted: completedSteps,
totalSteps: flow.steps.length,
mode: flow.mode,
};
}
getAllFlowSummaries(): FlowSummary[] {
return [...this.flows.keys()]
.map((id) => this.getFlowSummary(id))
.filter((s): s is FlowSummary => s !== null);
}
getMetrics(): FlowMetrics {
return { ...this.metrics };
}
cancelFlow(flowId: string): boolean {
const flow = this.flows.get(flowId);
if (!flow || flow.status !== 'running') return false;
flow.status = 'failed';
flow.completedAt = new Date().toISOString();
const currentStep = flow.steps[flow.currentStepIndex];
if (currentStep && currentStep.status === 'running') {
currentStep.status = 'failed';
currentStep.error = 'Cancelled';
currentStep.completedAt = new Date().toISOString();
}
return true;
}
clearCompletedFlows(): number {
let cleared = 0;
for (const [id, flow] of this.flows) {
if (flow.status === 'completed' || flow.status === 'failed') {
this.flows.delete(id);
cleared++;
}
}
return cleared;
}
private calculateDuration(start?: string, end?: string): number {
if (!start || !end) return 0;
return new Date(end).getTime() - new Date(start).getTime();
}
private updateAverageDuration(duration: number): void {
const completedCount = this.metrics.completedFlows;
this.metrics.averageDuration =
(this.metrics.averageDuration * (completedCount - 1) + duration) / completedCount;
}
private updateStepMetrics(stepName: string, success: boolean, duration: number): void {
const existing = this.metrics.stepMetrics.get(stepName) || {
executionCount: 0,
successCount: 0,
failureCount: 0,
averageDuration: 0,
};
existing.executionCount++;
if (success) {
existing.successCount++;
} else {
existing.failureCount++;
}
existing.averageDuration =
(existing.averageDuration * (existing.executionCount - 1) + duration) /
existing.executionCount;
this.metrics.stepMetrics.set(stepName, existing);
}
private delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
}
export function createExecutionManager(config?: ExecutionFlowConfig): ExecutionManager {
return new ExecutionManager(config);
}
+257
View File
@@ -0,0 +1,257 @@
import { existsSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { homedir } from 'node:os';
import type { ConnectorMapping } from '../connectors/types.js';
import { suggestMappingsFromMcp } from '../connectors/utils.js';
export type ExecutionMode = 'standalone' | 'enhanced';
export interface ModeDetectionResult {
mode: ExecutionMode;
availableServers: string[];
missingServers: string[];
capabilities: ModeCapabilities;
connectorMappings: ConnectorMapping[];
}
export interface ModeCapabilities {
canAccessCRM: boolean;
canSendEmails: boolean;
canAccessCalendar: boolean;
canAccessDocs: boolean;
canQueryData: boolean;
canEnrichData: boolean;
canSendNotifications: boolean;
canUseAI: boolean;
hasChat: boolean;
hasStorage: boolean;
hasSearch: boolean;
hasAnalytics: boolean;
}
export interface ExecutionModeConfig {
requiredServers?: string[];
optionalServers?: string[];
mcpConfigPaths?: string[];
fallbackToStandalone?: boolean;
}
const DEFAULT_MCP_CONFIG_PATHS = [
join(homedir(), '.mcp.json'),
join(homedir(), '.config', 'claude', 'mcp.json'),
'.mcp.json',
'mcp.json',
];
const CAPABILITY_SERVERS: Record<keyof ModeCapabilities, string[]> = {
canAccessCRM: ['salesforce', 'hubspot', 'pipedrive', 'zoho'],
canSendEmails: ['gmail', 'outlook', 'sendgrid', 'mailgun', 'ses'],
canAccessCalendar: ['google-calendar', 'outlook', 'calendly'],
canAccessDocs: ['notion', 'confluence', 'google-docs', 'coda'],
canQueryData: ['postgres', 'mysql', 'bigquery', 'snowflake', 'supabase', 'mongodb'],
canEnrichData: ['clearbit', 'zoominfo', 'apollo', 'hunter'],
canSendNotifications: ['slack', 'teams', 'discord', 'pagerduty', 'opsgenie'],
canUseAI: ['openai', 'anthropic', 'ollama', 'huggingface'],
hasChat: ['slack', 'teams', 'discord'],
hasStorage: ['s3', 'gcs', 'dropbox', 'drive', 'box'],
hasSearch: ['glean', 'elastic', 'algolia', 'typesense'],
hasAnalytics: ['mixpanel', 'amplitude', 'posthog', 'segment'],
};
export function detectExecutionMode(config: ExecutionModeConfig = {}): ModeDetectionResult {
const { requiredServers = [], optionalServers = [], mcpConfigPaths, fallbackToStandalone = true } = config;
const configPaths = mcpConfigPaths || DEFAULT_MCP_CONFIG_PATHS;
const availableServers = detectMcpServers(configPaths);
const missingRequired = requiredServers.filter(
(server) => !availableServers.some((s) => s.toLowerCase().includes(server.toLowerCase()))
);
const capabilities = detectCapabilities(availableServers);
const connectorMappings = suggestMappingsFromMcp(availableServers);
let mode: ExecutionMode = 'standalone';
if (missingRequired.length === 0 && availableServers.length > 0) {
mode = 'enhanced';
} else if (missingRequired.length > 0 && !fallbackToStandalone) {
throw new Error(`Missing required MCP servers: ${missingRequired.join(', ')}`);
}
const missingOptional = optionalServers.filter(
(server) => !availableServers.some((s) => s.toLowerCase().includes(server.toLowerCase()))
);
return {
mode,
availableServers,
missingServers: [...missingRequired, ...missingOptional],
capabilities,
connectorMappings,
};
}
function detectMcpServers(configPaths: string[]): string[] {
const servers: string[] = [];
for (const configPath of configPaths) {
try {
const fullPath = configPath.startsWith('~')
? configPath.replace('~', process.env.HOME || '')
: configPath;
if (existsSync(fullPath)) {
const content = readFileSync(fullPath, 'utf-8');
const config = JSON.parse(content);
if (config.mcpServers) {
servers.push(...Object.keys(config.mcpServers));
}
if (config.servers) {
servers.push(...Object.keys(config.servers));
}
}
} catch {
continue;
}
}
return [...new Set(servers)];
}
function detectCapabilities(servers: string[]): ModeCapabilities {
const serverLower = servers.map((s) => s.toLowerCase());
const capabilities: ModeCapabilities = {
canAccessCRM: false,
canSendEmails: false,
canAccessCalendar: false,
canAccessDocs: false,
canQueryData: false,
canEnrichData: false,
canSendNotifications: false,
canUseAI: false,
hasChat: false,
hasStorage: false,
hasSearch: false,
hasAnalytics: false,
};
for (const [capability, keywords] of Object.entries(CAPABILITY_SERVERS)) {
for (const keyword of keywords) {
if (serverLower.some((server) => server.includes(keyword))) {
capabilities[capability as keyof ModeCapabilities] = true;
break;
}
}
}
return capabilities;
}
export function getModeDescription(result: ModeDetectionResult): string {
const lines: string[] = [];
lines.push(`Mode: ${result.mode.toUpperCase()}`);
lines.push('');
if (result.availableServers.length > 0) {
lines.push(`Available MCP Servers (${result.availableServers.length}):`);
for (const server of result.availableServers.slice(0, 10)) {
lines.push(` - ${server}`);
}
if (result.availableServers.length > 10) {
lines.push(` ... and ${result.availableServers.length - 10} more`);
}
lines.push('');
}
if (result.missingServers.length > 0) {
lines.push(`Missing Servers:`);
for (const server of result.missingServers) {
lines.push(` - ${server}`);
}
lines.push('');
}
lines.push('Capabilities:');
const capabilityLabels: Record<keyof ModeCapabilities, string> = {
canAccessCRM: 'CRM Access',
canSendEmails: 'Email',
canAccessCalendar: 'Calendar',
canAccessDocs: 'Documents',
canQueryData: 'Data Queries',
canEnrichData: 'Data Enrichment',
canSendNotifications: 'Notifications',
canUseAI: 'AI Services',
hasChat: 'Chat',
hasStorage: 'Storage',
hasSearch: 'Search',
hasAnalytics: 'Analytics',
};
for (const [key, label] of Object.entries(capabilityLabels)) {
const enabled = result.capabilities[key as keyof ModeCapabilities];
lines.push(` ${enabled ? '✓' : '✗'} ${label}`);
}
if (result.connectorMappings.length > 0) {
lines.push('');
lines.push('Suggested Connector Mappings:');
for (const mapping of result.connectorMappings.slice(0, 5)) {
lines.push(` ${mapping.placeholder} → ${mapping.tool}`);
}
}
return lines.join('\n');
}
export function requireEnhancedMode(result: ModeDetectionResult): void {
if (result.mode !== 'enhanced') {
throw new Error(
`This skill requires enhanced mode with MCP servers. Missing: ${result.missingServers.join(', ')}`
);
}
}
export function requireCapability(
result: ModeDetectionResult,
capability: keyof ModeCapabilities
): void {
if (!result.capabilities[capability]) {
throw new Error(`This skill requires the ${capability} capability which is not available.`);
}
}
export function getStandaloneAlternative(capability: keyof ModeCapabilities): string {
const alternatives: Record<keyof ModeCapabilities, string> = {
canAccessCRM: 'Use a local JSON file to store contact information',
canSendEmails: 'Output email content to the terminal for manual sending',
canAccessCalendar: 'Use a local iCal file for scheduling',
canAccessDocs: 'Use local markdown files for documentation',
canQueryData: 'Use a local SQLite database or JSON files',
canEnrichData: 'Skip enrichment or use cached data',
canSendNotifications: 'Output notifications to the terminal',
canUseAI: 'Use simpler rule-based logic or skip AI features',
hasChat: 'Output messages to the terminal',
hasStorage: 'Use the local filesystem',
hasSearch: 'Use grep or local file search',
hasAnalytics: 'Skip analytics or use local logging',
};
return alternatives[capability];
}
export function createModeAwareExecutor<T>(
enhancedFn: () => Promise<T>,
standaloneFn: () => Promise<T>,
modeResult: ModeDetectionResult
): () => Promise<T> {
return async () => {
if (modeResult.mode === 'enhanced') {
return enhancedFn();
}
return standaloneFn();
};
}
+106
View File
@@ -0,0 +1,106 @@
import { z } from 'zod';
export const ExecutionStepStatusSchema = z.enum([
'pending',
'running',
'completed',
'failed',
'skipped',
]);
export type ExecutionStepStatus = z.infer<typeof ExecutionStepStatusSchema>;
export const ExecutionStepSchema = z.object({
id: z.string(),
name: z.string(),
description: z.string().optional(),
status: ExecutionStepStatusSchema.default('pending'),
startedAt: z.string().datetime().optional(),
completedAt: z.string().datetime().optional(),
duration: z.number().optional(),
input: z.record(z.unknown()).optional(),
output: z.record(z.unknown()).optional(),
error: z.string().optional(),
retryCount: z.number().default(0),
metadata: z.record(z.unknown()).optional(),
});
export type ExecutionStep = z.infer<typeof ExecutionStepSchema>;
export const ExecutionFlowSchema = z.object({
id: z.string(),
skillName: z.string(),
version: z.number().default(1),
steps: z.array(ExecutionStepSchema),
currentStepIndex: z.number().default(-1),
status: ExecutionStepStatusSchema.default('pending'),
startedAt: z.string().datetime().optional(),
completedAt: z.string().datetime().optional(),
totalDuration: z.number().optional(),
context: z.record(z.unknown()).optional(),
mode: z.enum(['standalone', 'enhanced']).default('standalone'),
mcpServers: z.array(z.string()).optional(),
});
export type ExecutionFlow = z.infer<typeof ExecutionFlowSchema>;
export interface ExecutionFlowConfig {
maxRetries?: number;
retryDelay?: number;
timeout?: number;
onStepStart?: (step: ExecutionStep, flow: ExecutionFlow) => void;
onStepComplete?: (step: ExecutionStep, flow: ExecutionFlow) => void;
onStepError?: (step: ExecutionStep, error: Error, flow: ExecutionFlow) => void;
onFlowComplete?: (flow: ExecutionFlow) => void;
}
export interface StepDefinition {
name: string;
description?: string;
execute: (
input: Record<string, unknown>,
context: ExecutionContext
) => Promise<Record<string, unknown>>;
rollback?: (context: ExecutionContext) => Promise<void>;
condition?: (context: ExecutionContext) => boolean;
retryable?: boolean;
timeout?: number;
}
export interface ExecutionContext {
flow: ExecutionFlow;
stepIndex: number;
previousOutput?: Record<string, unknown>;
mcpTools?: string[];
isEnhanced: boolean;
}
export interface FlowSummary {
id: string;
skillName: string;
status: ExecutionStepStatus;
progress: number;
currentStep?: string;
startedAt?: string;
completedAt?: string;
totalDuration?: number;
stepsCompleted: number;
totalSteps: number;
mode: 'standalone' | 'enhanced';
}
export interface FlowMetrics {
totalFlows: number;
completedFlows: number;
failedFlows: number;
averageDuration: number;
stepMetrics: Map<
string,
{
executionCount: number;
successCount: number;
failureCount: number;
averageDuration: number;
}
>;
}
+15
View File
@@ -89,3 +89,18 @@ export * from './profiles/index.js';
// Coding Guidelines System
export * from './guidelines/index.js';
// Skill Tree (Hierarchical Taxonomy - Phase 21)
export * from './tree/index.js';
// Reasoning Engine (LLM-based Tree Search - Phase 21)
export * from './reasoning/index.js';
// Connectors (Tool-Agnostic Placeholders - Phase 21)
export * from './connectors/index.js';
// Execution Flow (Step Tracking & Metrics - Phase 21)
export * from './execution/index.js';
// Hybrid Search (QMD-Inspired Vector + Keyword Search)
export * from './search/index.js';
@@ -0,0 +1,194 @@
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { WellKnownProvider, generateWellKnownIndex, generateWellKnownStructure, calculateBaseSkillsUrl } from '../wellknown.js';
import { existsSync, rmSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import { randomUUID } from 'node:crypto';
describe('WellKnownProvider', () => {
const provider = new WellKnownProvider();
describe('matches', () => {
it('should match https URLs not on github/gitlab/bitbucket', () => {
expect(provider.matches('https://example.com')).toBe(true);
expect(provider.matches('https://skills.vercel.app')).toBe(true);
expect(provider.matches('https://my-skills.com/api')).toBe(true);
});
it('should not match github.com URLs', () => {
expect(provider.matches('https://github.com/owner/repo')).toBe(false);
});
it('should not match gitlab.com URLs', () => {
expect(provider.matches('https://gitlab.com/owner/repo')).toBe(false);
});
it('should not match bitbucket.org URLs', () => {
expect(provider.matches('https://bitbucket.org/owner/repo')).toBe(false);
});
it('should not match local paths', () => {
expect(provider.matches('./my-skills')).toBe(false);
expect(provider.matches('/absolute/path')).toBe(false);
expect(provider.matches('~/home/skills')).toBe(false);
});
it('should not match shorthand GitHub format', () => {
expect(provider.matches('owner/repo')).toBe(false);
});
});
describe('parseSource', () => {
it('should parse URL hostname and path', () => {
const result = provider.parseSource('https://example.com/api');
expect(result).toEqual({ owner: 'example.com', repo: 'api' });
});
it('should handle root URLs', () => {
const result = provider.parseSource('https://skills.example.com');
expect(result).toEqual({ owner: 'skills.example.com', repo: 'skills' });
});
it('should return null for invalid URLs', () => {
const result = provider.parseSource('not-a-url');
expect(result).toBeNull();
});
});
describe('type and name', () => {
it('should have correct type', () => {
expect(provider.type).toBe('wellknown');
});
it('should have correct name', () => {
expect(provider.name).toBe('Well-Known');
});
});
});
describe('calculateBaseSkillsUrl', () => {
it('should handle index.json URL correctly', () => {
const result = calculateBaseSkillsUrl('https://example.com/.well-known/skills/index.json');
expect(result).toBe('https://example.com/.well-known/skills');
});
it('should handle skills.json URL correctly without duplicating .well-known', () => {
const result = calculateBaseSkillsUrl('https://example.com/.well-known/skills.json');
expect(result).toBe('https://example.com/.well-known/skills');
});
it('should handle nested paths with index.json', () => {
const result = calculateBaseSkillsUrl('https://cdn.example.com/v1/.well-known/skills/index.json');
expect(result).toBe('https://cdn.example.com/v1/.well-known/skills');
});
it('should handle nested paths with skills.json', () => {
const result = calculateBaseSkillsUrl('https://cdn.example.com/v1/.well-known/skills.json');
expect(result).toBe('https://cdn.example.com/v1/.well-known/skills');
});
});
describe('generateWellKnownIndex', () => {
it('should generate valid index structure', () => {
const skills = [
{ name: 'test-skill', description: 'A test skill', files: ['SKILL.md', 'README.md'] },
{ name: 'another-skill', description: 'Another skill', files: ['SKILL.md'] },
];
const index = generateWellKnownIndex(skills);
expect(index.version).toBe('1.0');
expect(index.skills).toHaveLength(2);
expect(index.skills[0]).toEqual({
name: 'test-skill',
description: 'A test skill',
files: ['SKILL.md', 'README.md'],
});
expect(index.skills[1]).toEqual({
name: 'another-skill',
description: 'Another skill',
files: ['SKILL.md'],
});
});
it('should handle empty skills array', () => {
const index = generateWellKnownIndex([]);
expect(index.version).toBe('1.0');
expect(index.skills).toEqual([]);
});
});
describe('generateWellKnownStructure', () => {
let tempDir: string;
beforeEach(() => {
tempDir = join(tmpdir(), `skillkit-test-${randomUUID()}`);
});
afterEach(() => {
if (existsSync(tempDir)) {
rmSync(tempDir, { recursive: true, force: true });
}
});
it('should create well-known directory structure', () => {
const skills = [
{
name: 'test-skill',
description: 'A test skill',
content: '---\nname: test-skill\ndescription: A test skill\n---\n\n# Test Skill\n\nContent here.',
},
];
const result = generateWellKnownStructure(tempDir, skills);
expect(existsSync(join(tempDir, '.well-known', 'skills'))).toBe(true);
expect(existsSync(result.indexPath)).toBe(true);
expect(result.skillPaths).toContain(join(tempDir, '.well-known', 'skills', 'test-skill', 'SKILL.md'));
const index = JSON.parse(readFileSync(result.indexPath, 'utf-8'));
expect(index.skills[0].name).toBe('test-skill');
expect(index.skills[0].files).toContain('SKILL.md');
});
it('should include additional files', () => {
const skills = [
{
name: 'test-skill',
description: 'Test',
content: '# Test',
additionalFiles: {
'README.md': '# README',
'config.json': '{"key": "value"}',
},
},
];
const result = generateWellKnownStructure(tempDir, skills);
expect(result.skillPaths).toContain(join(tempDir, '.well-known', 'skills', 'test-skill', 'README.md'));
expect(result.skillPaths).toContain(join(tempDir, '.well-known', 'skills', 'test-skill', 'config.json'));
const readme = readFileSync(join(tempDir, '.well-known', 'skills', 'test-skill', 'README.md'), 'utf-8');
expect(readme).toBe('# README');
const index = JSON.parse(readFileSync(result.indexPath, 'utf-8'));
expect(index.skills[0].files).toContain('README.md');
expect(index.skills[0].files).toContain('config.json');
});
it('should handle multiple skills', () => {
const skills = [
{ name: 'skill-one', description: 'First', content: '# Skill One' },
{ name: 'skill-two', description: 'Second', content: '# Skill Two' },
];
const result = generateWellKnownStructure(tempDir, skills);
expect(existsSync(join(tempDir, '.well-known', 'skills', 'skill-one', 'SKILL.md'))).toBe(true);
expect(existsSync(join(tempDir, '.well-known', 'skills', 'skill-two', 'SKILL.md'))).toBe(true);
const index = JSON.parse(readFileSync(result.indexPath, 'utf-8'));
expect(index.skills).toHaveLength(2);
});
});
+3
View File
@@ -4,17 +4,20 @@ import { GitHubProvider } from './github.js';
import { GitLabProvider } from './gitlab.js';
import { BitbucketProvider } from './bitbucket.js';
import { LocalProvider } from './local.js';
import { WellKnownProvider } from './wellknown.js';
export * from './base.js';
export * from './github.js';
export * from './gitlab.js';
export * from './bitbucket.js';
export * from './local.js';
export * from './wellknown.js';
const providers: GitProviderAdapter[] = [
new LocalProvider(),
new GitLabProvider(),
new BitbucketProvider(),
new WellKnownProvider(),
new GitHubProvider(),
];
+248
View File
@@ -0,0 +1,248 @@
import { existsSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
import { join, basename, resolve, sep } from 'node:path';
import { tmpdir } from 'node:os';
import { randomUUID } from 'node:crypto';
import type { GitProviderAdapter, CloneOptions } from './base.js';
import { isGitUrl, isLocalPath } from './base.js';
import type { GitProvider, CloneResult } from '../types.js';
function sanitizeSkillName(name: string): string | null {
if (!name || typeof name !== 'string') return null;
const base = basename(name);
if (base !== name || name.includes('..') || name.includes('/') || name.includes('\\')) {
return null;
}
if (!/^[a-zA-Z0-9._-]+$/.test(name)) {
return null;
}
return name;
}
export interface WellKnownSkill {
name: string;
description?: string;
files: string[];
}
export interface WellKnownIndex {
version?: string;
skills: WellKnownSkill[];
}
export class WellKnownProvider implements GitProviderAdapter {
readonly type: GitProvider = 'wellknown';
readonly name = 'Well-Known';
readonly baseUrl = '';
parseSource(source: string): { owner: string; repo: string; subpath?: string } | null {
try {
const url = new URL(source);
return { owner: url.hostname, repo: url.pathname.replace(/^\//, '') || 'skills' };
} catch {
return null;
}
}
matches(source: string): boolean {
if (isLocalPath(source)) return false;
if (!isGitUrl(source)) return false;
if (source.includes('github.com')) return false;
if (source.includes('gitlab.com')) return false;
if (source.includes('bitbucket.org')) return false;
try {
new URL(source);
return true;
} catch {
return false;
}
}
getCloneUrl(_owner: string, _repo: string): string {
return '';
}
getSshUrl(_owner: string, _repo: string): string {
return '';
}
async clone(source: string, _targetDir: string, _options: CloneOptions = {}): Promise<CloneResult> {
const tempDir = join(tmpdir(), `skillkit-wellknown-${randomUUID()}`);
try {
mkdirSync(tempDir, { recursive: true });
const baseUrl = source.replace(/\/$/, '');
const indexUrls = [
`${baseUrl}/.well-known/skills/index.json`,
`${baseUrl}/.well-known/skills.json`,
];
let index: WellKnownIndex | null = null;
let foundUrl = '';
for (const url of indexUrls) {
try {
const response = await fetch(url);
if (response.ok) {
index = await response.json() as WellKnownIndex;
foundUrl = url;
break;
}
} catch {
continue;
}
}
if (!index || !index.skills || index.skills.length === 0) {
return {
success: false,
error: `No skills found at ${baseUrl}/.well-known/skills/index.json`,
};
}
const skills: string[] = [];
const discoveredSkills: Array<{ name: string; dirName: string; path: string }> = [];
const baseSkillsUrl = calculateBaseSkillsUrl(foundUrl);
for (const skill of index.skills) {
const safeName = sanitizeSkillName(skill.name);
if (!safeName) {
continue;
}
const skillDir = join(tempDir, safeName);
const resolvedSkillDir = resolve(skillDir);
const resolvedTempDir = resolve(tempDir);
if (!resolvedSkillDir.startsWith(resolvedTempDir + sep) && resolvedSkillDir !== resolvedTempDir) {
continue;
}
mkdirSync(skillDir, { recursive: true });
let hasSkillMd = false;
for (const file of skill.files) {
const fileUrl = `${baseSkillsUrl}/${encodeURIComponent(skill.name)}/${encodeURIComponent(file)}`;
try {
const response = await fetch(fileUrl);
if (response.ok) {
const content = await response.text();
const safeFileName = basename(file);
writeFileSync(join(skillDir, safeFileName), content);
if (file === 'SKILL.md' || file.endsWith('/SKILL.md')) {
hasSkillMd = true;
}
}
} catch {
continue;
}
}
if (hasSkillMd) {
skills.push(safeName);
discoveredSkills.push({
name: safeName,
dirName: safeName,
path: skillDir,
});
}
}
if (skills.length === 0) {
rmSync(tempDir, { recursive: true, force: true });
return {
success: false,
error: 'No valid skills found (skills must contain SKILL.md)',
};
}
return {
success: true,
path: tempDir,
tempRoot: tempDir,
skills,
discoveredSkills,
};
} catch (error) {
if (existsSync(tempDir)) {
rmSync(tempDir, { recursive: true, force: true });
}
const message = error instanceof Error ? error.message : String(error);
return { success: false, error: `Failed to fetch skills: ${message}` };
}
}
}
export function calculateBaseSkillsUrl(foundUrl: string): string {
return foundUrl.endsWith('/index.json')
? foundUrl.replace('/index.json', '')
: foundUrl.replace('/skills.json', '/skills');
}
export function generateWellKnownIndex(skills: Array<{ name: string; description?: string; files: string[] }>): WellKnownIndex {
return {
version: '1.0',
skills: skills.map(s => ({
name: s.name,
description: s.description,
files: s.files,
})),
};
}
export function generateWellKnownStructure(
outputDir: string,
skills: Array<{ name: string; description?: string; content: string; additionalFiles?: Record<string, string> }>
): { indexPath: string; skillPaths: string[] } {
const wellKnownDir = join(outputDir, '.well-known', 'skills');
const resolvedWellKnownDir = resolve(wellKnownDir);
mkdirSync(wellKnownDir, { recursive: true });
const indexSkills: WellKnownSkill[] = [];
const skillPaths: string[] = [];
for (const skill of skills) {
const safeName = sanitizeSkillName(skill.name);
if (!safeName) {
continue;
}
const skillDir = join(wellKnownDir, safeName);
const resolvedSkillDir = resolve(skillDir);
if (!resolvedSkillDir.startsWith(resolvedWellKnownDir + sep) && resolvedSkillDir !== resolvedWellKnownDir) {
continue;
}
mkdirSync(skillDir, { recursive: true });
writeFileSync(join(skillDir, 'SKILL.md'), skill.content);
skillPaths.push(join(skillDir, 'SKILL.md'));
const files = ['SKILL.md'];
if (skill.additionalFiles) {
for (const [filename, content] of Object.entries(skill.additionalFiles)) {
writeFileSync(join(skillDir, filename), content);
files.push(filename);
skillPaths.push(join(skillDir, filename));
}
}
indexSkills.push({
name: skill.name,
description: skill.description,
files,
});
}
const index = generateWellKnownIndex(indexSkills);
const indexPath = join(wellKnownDir, 'index.json');
writeFileSync(indexPath, JSON.stringify(index, null, 2));
return { indexPath, skillPaths };
}
+618
View File
@@ -0,0 +1,618 @@
import type { SkillSummary, ProjectProfile } from '../recommend/types.js';
import type { SkillTree, TreeNode } from '../tree/types.js';
import { TreeGenerator } from '../tree/generator.js';
import {
type ReasoningConfig,
type TreeSearchQuery,
type TreeSearchResult,
type TreeReasoningResult,
type ExplainedRecommendation,
type ExplainedMatch,
type SearchPlan,
type ReasoningCacheEntry,
type ReasoningEngineStats,
DEFAULT_REASONING_CONFIG,
} from './types.js';
import {
buildSearchPlanPrompt,
buildExplanationPrompt,
extractJsonFromResponse,
validateSearchPlan,
} from './prompts.js';
import { CATEGORY_TAXONOMY } from '../tree/types.js';
const CACHE_TTL_MS = 5 * 60 * 1000;
const MAX_CACHE_SIZE = 100;
export class ReasoningEngine {
private config: ReasoningConfig;
private skillMap: Map<string, SkillSummary> = new Map();
private tree: SkillTree | null = null;
private cache: Map<string, ReasoningCacheEntry> = new Map();
private stats: ReasoningEngineStats = {
totalQueries: 0,
cacheHits: 0,
cacheMisses: 0,
averageProcessingTimeMs: 0,
averageResultsPerQuery: 0,
};
constructor(config?: Partial<ReasoningConfig>) {
this.config = { ...DEFAULT_REASONING_CONFIG, ...config };
}
loadSkills(skills: SkillSummary[]): void {
this.skillMap.clear();
for (const skill of skills) {
this.skillMap.set(skill.name, skill);
}
}
loadTree(tree: SkillTree): void {
this.tree = tree;
}
generateTree(skills: SkillSummary[]): void {
const generator = new TreeGenerator();
this.tree = generator.generateTree(skills);
this.loadSkills(skills);
}
async search(query: TreeSearchQuery): Promise<TreeReasoningResult> {
const startTime = Date.now();
this.stats.totalQueries++;
const cacheKey = this.getCacheKey(query);
const cached = this.getFromCache(cacheKey);
if (cached) {
this.stats.cacheHits++;
return {
query: query.query,
results: cached.results,
exploredPaths: [],
reasoning: 'Retrieved from cache',
totalNodesVisited: 0,
processingTimeMs: Date.now() - startTime,
};
}
this.stats.cacheMisses++;
const maxResults = query.maxResults ?? 10;
const minConfidence = query.minConfidence ?? 30;
const plan = await this.createSearchPlan(query);
const results: TreeSearchResult[] = [];
const exploredPaths: string[][] = [];
let totalNodesVisited = 0;
if (this.tree) {
const traversalResult = await this.traverseTree(query, plan);
results.push(...traversalResult.results);
exploredPaths.push(...traversalResult.exploredPaths);
totalNodesVisited = traversalResult.nodesVisited;
} else {
const fallbackResults = this.fallbackSearch(query);
results.push(...fallbackResults);
}
const filteredResults = results
.filter((r) => r.confidence >= minConfidence)
.sort((a, b) => b.confidence - a.confidence)
.slice(0, maxResults);
const processingTimeMs = Date.now() - startTime;
this.updateStats(processingTimeMs, filteredResults.length);
this.addToCache(cacheKey, filteredResults);
return {
query: query.query,
results: filteredResults,
exploredPaths,
reasoning: this.buildReasoningSummary(plan, filteredResults),
totalNodesVisited,
processingTimeMs,
};
}
async explain(
skill: SkillSummary,
score: number,
profile: ProjectProfile
): Promise<ExplainedRecommendation> {
const treePath = this.getSkillPath(skill.name);
const explanation = await this.generateExplanation(skill, score, profile);
return {
skill,
score,
reasoning: explanation,
treePath,
};
}
async explainBatch(
skills: Array<{ skill: SkillSummary; score: number }>,
profile: ProjectProfile
): Promise<ExplainedRecommendation[]> {
const results: ExplainedRecommendation[] = [];
for (const { skill, score } of skills) {
const explained = await this.explain(skill, score, profile);
results.push(explained);
}
return results;
}
private async createSearchPlan(query: TreeSearchQuery): Promise<SearchPlan> {
const categories = CATEGORY_TAXONOMY.map((c) => c.category);
if (this.config.provider === 'mock') {
return this.mockSearchPlan(query.query, query.context);
}
const prompt = buildSearchPlanPrompt(
query.query,
categories,
query.context
);
const response = await this.callLLM(prompt);
try {
const data = extractJsonFromResponse(response);
return validateSearchPlan(data);
} catch {
return this.mockSearchPlan(query.query, query.context);
}
}
private mockSearchPlan(query: string, context?: ProjectProfile): SearchPlan {
const queryLower = query.toLowerCase();
const keywords = queryLower
.split(/\s+/)
.filter((w) => w.length > 2);
const categoryScores: { category: string; score: number }[] = [];
for (const mapping of CATEGORY_TAXONOMY) {
let score = 0;
for (const keyword of keywords) {
if (mapping.category.toLowerCase().includes(keyword)) {
score += 30;
}
for (const tag of mapping.tags) {
if (tag.includes(keyword) || keyword.includes(tag)) {
score += 20;
}
}
for (const kw of mapping.keywords) {
if (kw.includes(keyword) || keyword.includes(kw)) {
score += 15;
}
}
}
if (context) {
for (const framework of context.stack.frameworks) {
if (mapping.tags.includes(framework.name.toLowerCase())) {
score += 10;
}
}
for (const language of context.stack.languages) {
if (mapping.tags.includes(language.name.toLowerCase())) {
score += 5;
}
}
}
if (score > 0) {
categoryScores.push({ category: mapping.category, score });
}
}
categoryScores.sort((a, b) => b.score - a.score);
const primaryCategories = categoryScores
.slice(0, 3)
.map((c) => c.category);
const secondaryCategories = categoryScores
.slice(3, 6)
.map((c) => c.category);
const filters: SearchPlan['filters'] = {
tags: keywords.filter((k) =>
CATEGORY_TAXONOMY.some((m) => m.tags.includes(k))
),
frameworks: context?.stack.frameworks.map((f) => f.name) || [],
languages: context?.stack.languages.map((l) => l.name) || [],
};
return {
primaryCategories: primaryCategories.length > 0 ? primaryCategories : ['Development'],
secondaryCategories,
keywords,
filters,
strategy: keywords.length <= 2 ? 'breadth-first' : 'targeted',
};
}
private async traverseTree(
query: TreeSearchQuery,
plan: SearchPlan
): Promise<{
results: TreeSearchResult[];
exploredPaths: string[][];
nodesVisited: number;
}> {
if (!this.tree) {
return { results: [], exploredPaths: [], nodesVisited: 0 };
}
const results: TreeSearchResult[] = [];
const exploredPaths: string[][] = [];
let nodesVisited = 0;
const relevantNodes = this.findRelevantNodes(plan);
for (const { node, path } of relevantNodes) {
nodesVisited++;
exploredPaths.push(path);
for (const skillName of node.skills) {
const skill = this.skillMap.get(skillName);
if (!skill) continue;
const matchResult = this.evaluateSkillMatch(query.query, skill, query.context);
if (matchResult.confidence >= (query.minConfidence ?? 30)) {
results.push({
skill,
path,
reasoning: matchResult.reasoning,
confidence: matchResult.confidence,
relevantSections: matchResult.relevantSections,
matchedKeywords: matchResult.matchedKeywords,
});
}
}
}
return { results, exploredPaths, nodesVisited };
}
private findRelevantNodes(
plan: SearchPlan
): Array<{ node: TreeNode; path: string[] }> {
if (!this.tree) return [];
const relevant: Array<{ node: TreeNode; path: string[]; priority: number }> = [];
const traverse = (node: TreeNode, path: string[], priorityBoost: number) => {
const nodeName = node.name.toLowerCase();
let priority = priorityBoost;
if (plan.primaryCategories.some((c) => c.toLowerCase() === nodeName)) {
priority += 100;
}
if (plan.secondaryCategories.some((c) => c.toLowerCase() === nodeName)) {
priority += 50;
}
for (const keyword of plan.keywords) {
if (nodeName.includes(keyword) || keyword.includes(nodeName)) {
priority += 25;
}
}
if (priority > 0 || node.depth === 0) {
relevant.push({ node, path: [...path, node.name], priority });
}
for (const child of node.children) {
traverse(child, [...path, node.name], priority > 0 ? priority * 0.5 : 0);
}
};
traverse(this.tree.rootNode, [], 0);
relevant.sort((a, b) => b.priority - a.priority);
return relevant.slice(0, 20).map(({ node, path }) => ({ node, path }));
}
private evaluateSkillMatch(
query: string,
skill: SkillSummary,
context?: ProjectProfile
): {
confidence: number;
reasoning: string;
relevantSections: string[];
matchedKeywords: string[];
} {
const queryTerms = query.toLowerCase().split(/\s+/);
const matchedKeywords: string[] = [];
let confidence = 0;
const nameLower = skill.name.toLowerCase();
for (const term of queryTerms) {
if (nameLower.includes(term)) {
confidence += 30;
matchedKeywords.push(term);
}
}
const descLower = (skill.description || '').toLowerCase();
for (const term of queryTerms) {
if (descLower.includes(term) && !matchedKeywords.includes(term)) {
confidence += 20;
matchedKeywords.push(term);
}
}
const tagsLower = (skill.tags || []).map((t) => t.toLowerCase());
for (const term of queryTerms) {
if (tagsLower.some((t) => t.includes(term) || term.includes(t))) {
if (!matchedKeywords.includes(term)) {
confidence += 25;
matchedKeywords.push(term);
}
}
}
if (context) {
for (const framework of context.stack.frameworks) {
const fwLower = framework.name.toLowerCase();
if (tagsLower.includes(fwLower) || nameLower.includes(fwLower)) {
confidence += 15;
}
}
for (const language of context.stack.languages) {
const langLower = language.name.toLowerCase();
if (tagsLower.includes(langLower) || nameLower.includes(langLower)) {
confidence += 10;
}
}
}
confidence = Math.min(100, confidence);
const relevantSections: string[] = [];
if (matchedKeywords.length > 0) {
relevantSections.push('description', 'tags');
}
const reasoning = matchedKeywords.length > 0
? `Matches keywords: ${matchedKeywords.join(', ')}`
: 'No direct keyword match';
return {
confidence,
reasoning,
relevantSections,
matchedKeywords,
};
}
private fallbackSearch(query: TreeSearchQuery): TreeSearchResult[] {
const results: TreeSearchResult[] = [];
for (const skill of this.skillMap.values()) {
const matchResult = this.evaluateSkillMatch(
query.query,
skill,
query.context
);
if (matchResult.confidence >= (query.minConfidence ?? 30)) {
results.push({
skill,
path: ['Uncategorized'],
reasoning: matchResult.reasoning,
confidence: matchResult.confidence,
relevantSections: matchResult.relevantSections,
matchedKeywords: matchResult.matchedKeywords,
});
}
}
return results;
}
private async generateExplanation(
skill: SkillSummary,
score: number,
profile: ProjectProfile
): Promise<ExplainedMatch> {
if (this.config.provider === 'mock') {
return this.mockExplanation(skill, score, profile);
}
const prompt = buildExplanationPrompt(skill, score, profile);
try {
const response = await this.callLLM(prompt);
const data = extractJsonFromResponse(response) as Partial<ExplainedMatch>;
return {
matchedBecause: data.matchedBecause || [],
relevantFor: data.relevantFor || [],
differentFrom: data.differentFrom || [],
confidence: data.confidence || this.scoreToConfidence(score),
};
} catch {
return this.mockExplanation(skill, score, profile);
}
}
private mockExplanation(
skill: SkillSummary,
score: number,
profile: ProjectProfile
): ExplainedMatch {
const matchedBecause: string[] = [];
const relevantFor: string[] = [];
const differentFrom: string[] = [];
const skillTags = skill.tags || [];
for (const framework of profile.stack.frameworks) {
const fwLower = framework.name.toLowerCase();
if (
skillTags.some((t) => t.toLowerCase().includes(fwLower)) ||
skill.name.toLowerCase().includes(fwLower)
) {
matchedBecause.push(`Uses ${framework.name}`);
}
}
for (const language of profile.stack.languages) {
const langLower = language.name.toLowerCase();
if (skillTags.some((t) => t.toLowerCase().includes(langLower))) {
matchedBecause.push(`Supports ${language.name}`);
}
}
if (matchedBecause.length === 0) {
matchedBecause.push(`Related tags: ${skillTags.slice(0, 3).join(', ')}`);
}
relevantFor.push(`Your ${profile.type || 'project'} project`);
if (profile.name) {
relevantFor.push(`Specifically for ${profile.name}`);
}
differentFrom.push('General-purpose alternatives');
return {
matchedBecause,
relevantFor,
differentFrom,
confidence: this.scoreToConfidence(score),
};
}
private scoreToConfidence(score: number): 'high' | 'medium' | 'low' {
if (score >= 70) return 'high';
if (score >= 50) return 'medium';
return 'low';
}
private getSkillPath(skillName: string): string[] {
if (!this.tree) return ['Uncategorized'];
const generator = new TreeGenerator();
const path = generator.getPath(this.tree, skillName);
return path || ['Uncategorized'];
}
private buildReasoningSummary(
plan: SearchPlan,
results: TreeSearchResult[]
): string {
const parts: string[] = [];
parts.push(`Search strategy: ${plan.strategy}`);
parts.push(`Primary categories: ${plan.primaryCategories.join(', ')}`);
if (plan.keywords.length > 0) {
parts.push(`Keywords extracted: ${plan.keywords.join(', ')}`);
}
parts.push(`Found ${results.length} matching skills`);
if (results.length > 0) {
const avgConfidence = results.reduce((sum, r) => sum + r.confidence, 0) / results.length;
parts.push(`Average confidence: ${Math.round(avgConfidence)}%`);
}
return parts.join('. ');
}
private async callLLM(prompt: string): Promise<string> {
return `Mock response for: ${prompt.slice(0, 50)}...`;
}
private getCacheKey(query: TreeSearchQuery): string {
return JSON.stringify({
query: query.query,
contextName: query.context?.name,
maxResults: query.maxResults,
minConfidence: query.minConfidence,
});
}
private getFromCache(key: string): ReasoningCacheEntry | null {
const entry = this.cache.get(key);
if (!entry) return null;
const now = Date.now();
if (now - entry.timestamp > entry.ttl) {
this.cache.delete(key);
return null;
}
return entry;
}
private addToCache(key: string, results: TreeSearchResult[]): void {
if (this.cache.size >= MAX_CACHE_SIZE) {
const oldestKey = this.cache.keys().next().value;
if (oldestKey) {
this.cache.delete(oldestKey);
}
}
this.cache.set(key, {
query: key,
results,
timestamp: Date.now(),
ttl: CACHE_TTL_MS,
});
}
private updateStats(processingTimeMs: number, resultCount: number): void {
const cacheMisses = this.stats.cacheMisses;
if (cacheMisses === 1) {
this.stats.averageProcessingTimeMs = processingTimeMs;
this.stats.averageResultsPerQuery = resultCount;
} else {
this.stats.averageProcessingTimeMs =
(this.stats.averageProcessingTimeMs * (cacheMisses - 1) + processingTimeMs) / cacheMisses;
this.stats.averageResultsPerQuery =
(this.stats.averageResultsPerQuery * (cacheMisses - 1) + resultCount) / cacheMisses;
}
}
getStats(): ReasoningEngineStats {
return { ...this.stats };
}
clearCache(): void {
this.cache.clear();
}
getTree(): SkillTree | null {
return this.tree;
}
}
export function createReasoningEngine(
config?: Partial<ReasoningConfig>
): ReasoningEngine {
return new ReasoningEngine(config);
}
+14
View File
@@ -0,0 +1,14 @@
export * from './types.js';
export * from './engine.js';
export * from './prompts.js';
export { ReasoningEngine, createReasoningEngine } from './engine.js';
export {
buildSearchPlanPrompt,
buildCategoryRelevancePrompt,
buildSkillMatchPrompt,
buildExplanationPrompt,
extractJsonFromResponse,
validateSearchPlan,
validateCategoryScore,
} from './prompts.js';
+227
View File
@@ -0,0 +1,227 @@
import type { TreeNode } from '../tree/types.js';
import type { ProjectProfile } from '../recommend/types.js';
import type { SearchPlan, CategoryScore } from './types.js';
export const SEARCH_PLANNING_PROMPT = `You are a skill discovery assistant. Given a user query and optional project context, create a search plan.
User Query: {{query}}
{{#if context}}
Project Context:
- Languages: {{context.languages}}
- Frameworks: {{context.frameworks}}
- Project Type: {{context.type}}
{{/if}}
Available Categories:
{{categories}}
Respond with a JSON search plan:
{
"primaryCategories": ["most relevant categories"],
"secondaryCategories": ["somewhat relevant categories"],
"keywords": ["extracted search terms"],
"filters": {
"tags": ["relevant tags"],
"frameworks": ["relevant frameworks"],
"languages": ["relevant languages"]
},
"strategy": "breadth-first" | "depth-first" | "targeted"
}`;
export const CATEGORY_RELEVANCE_PROMPT = `Rate the relevance of this category for the user's query.
Query: {{query}}
Category: {{category.name}}
Skills in category: {{category.skillCount}}
Subcategories: {{category.subcategories}}
Rate from 0-100 and explain briefly.
Respond as JSON: {"score": number, "reasoning": "brief explanation"}`;
export const SKILL_MATCH_PROMPT = `Evaluate how well this skill matches the user's query.
Query: {{query}}
Skill: {{skill.name}}
Description: {{skill.description}}
Tags: {{skill.tags}}
{{#if context}}
Project Stack:
- Languages: {{context.languages}}
- Frameworks: {{context.frameworks}}
{{/if}}
Evaluate match quality and explain why.
Respond as JSON:
{
"confidence": 0-100,
"matchedKeywords": ["words from query that match"],
"relevantSections": ["which parts of skill are relevant"],
"reasoning": "explanation of match quality"
}`;
export const EXPLANATION_PROMPT = `Explain why this skill was recommended for the user's project.
Skill: {{skill.name}}
Description: {{skill.description}}
Tags: {{skill.tags}}
Score: {{score}}
Project:
- Name: {{project.name}}
- Type: {{project.type}}
- Languages: {{project.languages}}
- Frameworks: {{project.frameworks}}
Explain the match clearly and concisely.
Respond as JSON:
{
"matchedBecause": ["specific match reasons"],
"relevantFor": ["how it helps this project"],
"differentFrom": ["what distinguishes it from alternatives"],
"confidence": "high" | "medium" | "low"
}`;
export function buildSearchPlanPrompt(
query: string,
categories: string[],
context?: ProjectProfile
): string {
let prompt = SEARCH_PLANNING_PROMPT
.replace('{{query}}', query)
.replace('{{categories}}', categories.join('\n'));
if (context) {
const languages = context.stack.languages.map(l => l.name).join(', ');
const frameworks = context.stack.frameworks.map(f => f.name).join(', ');
prompt = prompt
.replace('{{#if context}}', '')
.replace('{{/if}}', '')
.replace('{{context.languages}}', languages)
.replace('{{context.frameworks}}', frameworks)
.replace('{{context.type}}', context.type || 'unknown');
} else {
prompt = prompt.replace(/\{\{#if context\}\}[\s\S]*?\{\{\/if\}\}/g, '');
}
return prompt;
}
export function buildCategoryRelevancePrompt(
query: string,
node: TreeNode
): string {
const subcategories = node.children.map(c => c.name).join(', ');
return CATEGORY_RELEVANCE_PROMPT
.replace('{{query}}', query)
.replace('{{category.name}}', node.name)
.replace('{{category.skillCount}}', String(node.skillCount))
.replace('{{category.subcategories}}', subcategories || 'none');
}
export function buildSkillMatchPrompt(
query: string,
skill: { name: string; description?: string; tags?: string[] },
context?: ProjectProfile
): string {
let prompt = SKILL_MATCH_PROMPT
.replace('{{query}}', query)
.replace('{{skill.name}}', skill.name)
.replace('{{skill.description}}', skill.description || 'No description')
.replace('{{skill.tags}}', skill.tags?.join(', ') || 'No tags');
if (context) {
const languages = context.stack.languages.map(l => l.name).join(', ');
const frameworks = context.stack.frameworks.map(f => f.name).join(', ');
prompt = prompt
.replace('{{#if context}}', '')
.replace('{{/if}}', '')
.replace('{{context.languages}}', languages)
.replace('{{context.frameworks}}', frameworks);
} else {
prompt = prompt.replace(/\{\{#if context\}\}[\s\S]*?\{\{\/if\}\}/g, '');
}
return prompt;
}
export function buildExplanationPrompt(
skill: { name: string; description?: string; tags?: string[] },
score: number,
project: ProjectProfile
): string {
const languages = project.stack.languages.map(l => l.name).join(', ');
const frameworks = project.stack.frameworks.map(f => f.name).join(', ');
return EXPLANATION_PROMPT
.replace('{{skill.name}}', skill.name)
.replace('{{skill.description}}', skill.description || 'No description')
.replace('{{skill.tags}}', skill.tags?.join(', ') || 'No tags')
.replace('{{score}}', String(score))
.replace('{{project.name}}', project.name)
.replace('{{project.type}}', project.type || 'unknown')
.replace('{{project.languages}}', languages)
.replace('{{project.frameworks}}', frameworks);
}
export function extractJsonFromResponse(response: string): unknown {
const jsonMatch = response.match(/\{[\s\S]*\}/);
if (!jsonMatch) {
throw new Error('No JSON found in response');
}
try {
return JSON.parse(jsonMatch[0]);
} catch {
throw new Error('Invalid JSON in response');
}
}
export function validateSearchPlan(data: unknown): SearchPlan {
const plan = data as Record<string, unknown>;
return {
primaryCategories: Array.isArray(plan.primaryCategories)
? plan.primaryCategories
: [],
secondaryCategories: Array.isArray(plan.secondaryCategories)
? plan.secondaryCategories
: [],
keywords: Array.isArray(plan.keywords) ? plan.keywords : [],
filters: {
tags: Array.isArray((plan.filters as Record<string, unknown>)?.tags)
? (plan.filters as Record<string, unknown>).tags as string[]
: [],
frameworks: Array.isArray((plan.filters as Record<string, unknown>)?.frameworks)
? (plan.filters as Record<string, unknown>).frameworks as string[]
: [],
languages: Array.isArray((plan.filters as Record<string, unknown>)?.languages)
? (plan.filters as Record<string, unknown>).languages as string[]
: [],
},
strategy: ['breadth-first', 'depth-first', 'targeted'].includes(
plan.strategy as string
)
? (plan.strategy as 'breadth-first' | 'depth-first' | 'targeted')
: 'breadth-first',
};
}
export function validateCategoryScore(
data: unknown,
fallbackCategory = ''
): CategoryScore {
const score = data as Record<string, unknown>;
return {
category: typeof score.category === 'string' ? score.category : fallbackCategory,
score: typeof score.score === 'number' ? Math.min(100, Math.max(0, score.score)) : 0,
reasoning: typeof score.reasoning === 'string' ? score.reasoning : '',
};
}
+115
View File
@@ -0,0 +1,115 @@
import { z } from 'zod';
import type { SkillSummary, ProjectProfile } from '../recommend/types.js';
import type { TreeNode } from '../tree/types.js';
export const ReasoningProviderSchema = z.enum(['openai', 'anthropic', 'ollama', 'mock']);
export type ReasoningProvider = z.infer<typeof ReasoningProviderSchema>;
export interface ReasoningConfig {
provider: ReasoningProvider;
apiKey?: string;
model?: string;
maxTokens?: number;
temperature?: number;
baseUrl?: string;
}
export interface TreeSearchQuery {
query: string;
context?: ProjectProfile;
maxResults?: number;
minConfidence?: number;
searchDepth?: number;
}
export interface TreeTraversalStep {
node: TreeNode;
reasoning: string;
confidence: number;
action: 'explore' | 'skip' | 'select';
selectedSkills: string[];
}
export interface TreeSearchResult {
skill: SkillSummary;
path: string[];
reasoning: string;
confidence: number;
relevantSections: string[];
matchedKeywords: string[];
}
export interface TreeReasoningResult {
query: string;
results: TreeSearchResult[];
exploredPaths: string[][];
reasoning: string;
totalNodesVisited: number;
processingTimeMs: number;
}
export interface ExplainedMatch {
matchedBecause: string[];
relevantFor: string[];
differentFrom: string[];
confidence: 'high' | 'medium' | 'low';
}
export interface ExplainedRecommendation {
skill: SkillSummary;
score: number;
reasoning: ExplainedMatch;
treePath: string[];
}
export interface ReasoningPrompt {
name: string;
template: string;
variables: string[];
}
export interface LLMResponse {
content: string;
tokensUsed: number;
model: string;
}
export interface ReasoningCacheEntry {
query: string;
results: TreeSearchResult[];
timestamp: number;
ttl: number;
}
export const DEFAULT_REASONING_CONFIG: ReasoningConfig = {
provider: 'mock',
model: 'gpt-4o-mini',
maxTokens: 1000,
temperature: 0.3,
};
export interface CategoryScore {
category: string;
score: number;
reasoning: string;
}
export interface SearchPlan {
primaryCategories: string[];
secondaryCategories: string[];
keywords: string[];
filters: {
tags?: string[];
frameworks?: string[];
languages?: string[];
};
strategy: 'breadth-first' | 'depth-first' | 'targeted';
}
export interface ReasoningEngineStats {
totalQueries: number;
cacheHits: number;
cacheMisses: number;
averageProcessingTimeMs: number;
averageResultsPerQuery: number;
}
+233
View File
@@ -11,8 +11,11 @@ import type {
SearchOptions,
SearchResult,
FreshnessResult,
RecommendHybridSearchOptions,
RecommendHybridSearchResult,
} from './types.js';
import { DEFAULT_SCORING_WEIGHTS, TAG_TO_TECH, getTechTags } from './types.js';
import type { HybridSearchPipeline } from '../search/hybrid.js';
/**
* Recommendation engine for matching skills to project profiles
@@ -20,16 +23,39 @@ import { DEFAULT_SCORING_WEIGHTS, TAG_TO_TECH, getTechTags } from './types.js';
export class RecommendationEngine {
private weights: ScoringWeights;
private index: SkillIndex | null = null;
private hybridPipeline: HybridSearchPipeline | null = null;
constructor(weights?: Partial<ScoringWeights>) {
this.weights = { ...DEFAULT_SCORING_WEIGHTS, ...weights };
}
/**
* Initialize hybrid search pipeline for vector + keyword search
*/
async initHybridSearch(): Promise<void> {
const { createHybridSearchPipeline } = await import('../search/hybrid.js');
this.hybridPipeline = createHybridSearchPipeline();
if (this.index) {
this.hybridPipeline.loadSkillsIndex(this.index);
}
await this.hybridPipeline.initialize();
}
/**
* Check if hybrid search is available
*/
isHybridSearchAvailable(): boolean {
return this.hybridPipeline !== null && this.hybridPipeline.isInitialized();
}
/**
* Load skill index from cache or generate from local skills
*/
loadIndex(index: SkillIndex): void {
this.index = index;
if (this.hybridPipeline) {
this.hybridPipeline.loadSkillsIndex(index);
}
}
/**
@@ -603,6 +629,71 @@ export class RecommendationEngine {
return { relevance, matchedTerms, snippet };
}
/**
* Hybrid search combining vector embeddings and keyword matching
*/
async hybridSearch(options: RecommendHybridSearchOptions): Promise<RecommendHybridSearchResult[]> {
const { query, limit = 10, hybrid = true, enableExpansion = false, enableReranking = false, filters } = options;
if (!hybrid || !this.hybridPipeline) {
const basicResults = this.search({ query, limit, semantic: true, filters });
return basicResults.map((r) => ({
...r,
hybridScore: r.relevance / 100,
}));
}
const response = await this.hybridPipeline.search({
query,
limit,
enableExpansion,
enableReranking,
});
let results = response.results.map((r) => ({
skill: r.skill,
relevance: r.relevance,
matchedTerms: r.matchedTerms,
snippet: r.snippet,
hybridScore: r.hybridScore,
vectorSimilarity: r.vectorSimilarity,
keywordScore: r.keywordScore,
rrfScore: r.rrfScore,
expandedTerms: r.expandedTerms,
}));
if (filters?.tags && filters.tags.length > 0) {
results = results.filter((r) =>
r.skill.tags?.some((t) => filters.tags!.includes(t))
);
}
if (filters?.verified) {
results = results.filter((r) => r.skill.verified);
}
if (filters?.minScore) {
results = results.filter((r) => r.relevance >= filters.minScore!);
}
return results.slice(0, limit);
}
/**
* Build hybrid search index from skills
*/
async buildHybridIndex(
onProgress?: (progress: { phase: string; current: number; total: number; message?: string }) => void
): Promise<void> {
if (!this.index) {
throw new Error('No skill index loaded. Call loadIndex() first.');
}
if (!this.hybridPipeline) {
await this.initHybridSearch();
}
await this.hybridPipeline!.buildIndex(this.index.skills, onProgress);
}
/**
* Check freshness of installed skills against project dependencies
*
@@ -665,3 +756,145 @@ export function createRecommendationEngine(
): RecommendationEngine {
return new RecommendationEngine(weights);
}
/**
* Enhanced Recommendation Engine with reasoning support
*/
export class ReasoningRecommendationEngine extends RecommendationEngine {
private reasoningEngine: import('../reasoning/engine.js').ReasoningEngine | null = null;
private reasoningConfig?: import('../reasoning/types.js').ReasoningConfig;
constructor(
weights?: Partial<ScoringWeights>,
reasoningConfig?: Partial<import('../reasoning/types.js').ReasoningConfig>
) {
super(weights);
this.reasoningConfig = reasoningConfig as import('../reasoning/types.js').ReasoningConfig;
}
async initReasoning(): Promise<void> {
const { createReasoningEngine } = await import('../reasoning/engine.js');
this.reasoningEngine = createReasoningEngine(this.reasoningConfig);
const index = this.getIndex();
if (index) {
this.reasoningEngine.loadSkills(index.skills);
this.reasoningEngine.generateTree(index.skills);
}
}
async recommendWithReasoning(
profile: ProjectProfile,
options: import('./types.js').ReasoningRecommendOptions = {}
): Promise<import('./types.js').ReasoningRecommendationResult> {
const baseResult = this.recommend(profile, options);
if (!options.reasoning || !this.reasoningEngine) {
return {
...baseResult,
recommendations: baseResult.recommendations.map((r) => ({
...r,
treePath: [],
})),
};
}
const searchResult = await this.reasoningEngine.search({
query: this.buildQueryFromProfile(profile),
context: profile,
maxResults: options.limit ?? 10,
minConfidence: options.minScore ?? 30,
});
const enhancedRecommendations: import('./types.js').ExplainedScoredSkill[] = [];
for (const rec of baseResult.recommendations) {
const treeResult = searchResult.results.find(
(r) => r.skill.name === rec.skill.name
);
let explanation: import('./types.js').ExplainedMatchDetails | undefined;
if (options.explainResults) {
const explainedRec = await this.reasoningEngine.explain(
rec.skill,
rec.score,
profile
);
explanation = explainedRec.reasoning;
}
enhancedRecommendations.push({
...rec,
explanation,
treePath: treeResult?.path ?? [],
reasoningDetails: treeResult?.reasoning,
});
}
return {
...baseResult,
recommendations: enhancedRecommendations,
reasoningSummary: searchResult.reasoning,
searchPlan: {
primaryCategories: [],
secondaryCategories: [],
keywords: this.extractKeywords(profile),
strategy: 'breadth-first',
},
};
}
private buildQueryFromProfile(profile: ProjectProfile): string {
const parts: string[] = [];
if (profile.type) {
parts.push(profile.type);
}
for (const framework of profile.stack.frameworks.slice(0, 3)) {
parts.push(framework.name);
}
for (const language of profile.stack.languages.slice(0, 2)) {
parts.push(language.name);
}
return parts.join(' ');
}
private extractKeywords(profile: ProjectProfile): string[] {
const keywords: string[] = [];
if (profile.type) {
keywords.push(profile.type);
}
for (const framework of profile.stack.frameworks) {
keywords.push(framework.name.toLowerCase());
}
for (const language of profile.stack.languages) {
keywords.push(language.name.toLowerCase());
}
return [...new Set(keywords)];
}
getReasoningStats(): import('../reasoning/types.js').ReasoningEngineStats | null {
return this.reasoningEngine?.getStats() ?? null;
}
getSkillTree(): import('../tree/types.js').SkillTree | null {
return this.reasoningEngine?.getTree() ?? null;
}
}
/**
* Create an enhanced recommendation engine with reasoning support
*/
export function createReasoningRecommendationEngine(
weights?: Partial<ScoringWeights>
): ReasoningRecommendationEngine {
return new ReasoningRecommendationEngine(weights);
}
+6 -1
View File
@@ -14,7 +14,12 @@
export * from './types.js';
export * from './engine.js';
export * from './fetcher.js';
export { createRecommendationEngine, RecommendationEngine } from './engine.js';
export {
createRecommendationEngine,
RecommendationEngine,
ReasoningRecommendationEngine,
createReasoningRecommendationEngine,
} from './engine.js';
export {
fetchSkillsFromRepo,
buildSkillIndex,
+64
View File
@@ -284,3 +284,67 @@ export function getTechTags(techName: string): string[] {
}
return tags;
}
/**
* Enhanced recommendation options with reasoning support
*/
export interface ReasoningRecommendOptions extends RecommendOptions {
reasoning?: boolean;
explainResults?: boolean;
useTree?: boolean;
}
/**
* Explained match details
*/
export interface ExplainedMatchDetails {
matchedBecause: string[];
relevantFor: string[];
differentFrom: string[];
confidence: 'high' | 'medium' | 'low';
}
/**
* Enhanced scored skill with reasoning
*/
export interface ExplainedScoredSkill extends ScoredSkill {
explanation?: ExplainedMatchDetails;
treePath?: string[];
reasoningDetails?: string;
}
/**
* Enhanced recommendation result with reasoning
*/
export interface ReasoningRecommendationResult extends RecommendationResult {
recommendations: ExplainedScoredSkill[];
reasoningSummary?: string;
searchPlan?: {
primaryCategories: string[];
secondaryCategories: string[];
keywords: string[];
strategy: string;
};
}
/**
* Hybrid search options for RecommendationEngine
*/
export interface RecommendHybridSearchOptions extends SearchOptions {
hybrid?: boolean;
enableExpansion?: boolean;
enableReranking?: boolean;
semanticWeight?: number;
keywordWeight?: number;
}
/**
* Hybrid search result with additional metadata for RecommendationEngine
*/
export interface RecommendHybridSearchResult extends SearchResult {
hybridScore?: number;
vectorSimilarity?: number;
keywordScore?: number;
rrfScore?: number;
expandedTerms?: string[];
}
@@ -0,0 +1,217 @@
import { describe, it, expect, beforeEach } from 'vitest';
import { EmbeddingService } from '../embeddings.js';
import type { SkillSummary } from '../../recommend/types.js';
describe('EmbeddingService', () => {
describe('cosineSimilarity', () => {
let service: EmbeddingService;
beforeEach(() => {
service = new EmbeddingService();
});
it('should return 1 for identical vectors', () => {
const a = new Float32Array([1, 0, 0]);
const b = new Float32Array([1, 0, 0]);
const similarity = service.cosineSimilarity(a, b);
expect(similarity).toBeCloseTo(1, 5);
});
it('should return 0 for orthogonal vectors', () => {
const a = new Float32Array([1, 0, 0]);
const b = new Float32Array([0, 1, 0]);
const similarity = service.cosineSimilarity(a, b);
expect(similarity).toBeCloseTo(0, 5);
});
it('should return -1 for opposite vectors', () => {
const a = new Float32Array([1, 0, 0]);
const b = new Float32Array([-1, 0, 0]);
const similarity = service.cosineSimilarity(a, b);
expect(similarity).toBeCloseTo(-1, 5);
});
it('should handle normalized vectors', () => {
const a = new Float32Array([0.5, 0.5, 0.5, 0.5]);
const b = new Float32Array([0.5, 0.5, 0.5, 0.5]);
const similarity = service.cosineSimilarity(a, b);
expect(similarity).toBeCloseTo(1, 5);
});
it('should throw for mismatched dimensions', () => {
const a = new Float32Array([1, 0]);
const b = new Float32Array([1, 0, 0]);
expect(() => service.cosineSimilarity(a, b)).toThrow('Vector dimensions mismatch');
});
it('should return 0 for zero vectors', () => {
const a = new Float32Array([0, 0, 0]);
const b = new Float32Array([1, 0, 0]);
const similarity = service.cosineSimilarity(a, b);
expect(similarity).toBe(0);
});
});
describe('initialization state', () => {
it('should not be initialized by default', () => {
const service = new EmbeddingService();
expect(service.isInitialized()).toBe(false);
});
it('should return default dimensions', () => {
const service = new EmbeddingService();
expect(service.getDimensions()).toBe(768);
});
});
describe('skill text building', () => {
it('should handle skill with all fields', async () => {
const service = new EmbeddingService();
const skill: SkillSummary = {
name: 'test-skill',
description: 'A test skill',
tags: ['tag1', 'tag2'],
compatibility: {
frameworks: ['react'],
languages: ['typescript'],
libraries: ['lodash'],
},
popularity: 100,
quality: 80,
verified: true,
};
const buildSkillText = (s: SkillSummary): string => {
const parts: string[] = [s.name];
if (s.description) parts.push(s.description);
if (s.tags?.length) parts.push(s.tags.join(' '));
if (s.compatibility?.frameworks?.length) parts.push(s.compatibility.frameworks.join(' '));
if (s.compatibility?.languages?.length) parts.push(s.compatibility.languages.join(' '));
if (s.compatibility?.libraries?.length) parts.push(s.compatibility.libraries.join(' '));
return parts.join(' ').toLowerCase();
};
const text = buildSkillText(skill);
expect(text).toContain('test-skill');
expect(text).toContain('a test skill');
expect(text).toContain('tag1');
expect(text).toContain('react');
expect(text).toContain('typescript');
expect(text).toContain('lodash');
});
it('should handle skill with minimal fields', () => {
const skill: SkillSummary = {
name: 'minimal-skill',
tags: [],
popularity: 0,
quality: 50,
verified: false,
};
const buildSkillText = (s: SkillSummary): string => {
const parts: string[] = [s.name];
if (s.description) parts.push(s.description);
if (s.tags?.length) parts.push(s.tags.join(' '));
return parts.join(' ').toLowerCase();
};
const text = buildSkillText(skill);
expect(text).toBe('minimal-skill');
});
});
});
describe('EmbeddingService chunking', () => {
it('should split text into chunks', () => {
const chunkText = (
text: string,
config = { maxTokens: 100, overlapPercent: 15, minChunkSize: 10 }
) => {
const lines = text.split('\n');
const chunks: { content: string; startLine: number; endLine: number }[] = [];
const avgCharsPerToken = 4;
const maxCharsPerChunk = config.maxTokens * avgCharsPerToken;
const overlapChars = Math.floor(maxCharsPerChunk * (config.overlapPercent / 100));
let currentChunk: string[] = [];
let currentChars = 0;
let startLine = 0;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
const lineChars = line.length + 1;
if (currentChars + lineChars > maxCharsPerChunk && currentChunk.length > 0) {
const content = currentChunk.join('\n');
if (content.length >= config.minChunkSize) {
chunks.push({ content, startLine, endLine: i - 1 });
}
const overlapLines: string[] = [];
let overlapCharsCount = 0;
for (let j = currentChunk.length - 1; j >= 0 && overlapCharsCount < overlapChars; j--) {
overlapLines.unshift(currentChunk[j]);
overlapCharsCount += currentChunk[j].length + 1;
}
currentChunk = overlapLines;
currentChars = overlapCharsCount;
startLine = i - overlapLines.length;
}
currentChunk.push(line);
currentChars += lineChars;
}
if (currentChunk.length > 0) {
const content = currentChunk.join('\n');
if (content.length >= config.minChunkSize) {
chunks.push({ content, startLine, endLine: lines.length - 1 });
}
}
return chunks;
};
const longText = Array(50).fill('This is a line of text for testing.').join('\n');
const chunks = chunkText(longText);
expect(chunks.length).toBeGreaterThan(1);
chunks.forEach((chunk) => {
expect(chunk.content.length).toBeLessThanOrEqual(400);
expect(chunk.startLine).toBeGreaterThanOrEqual(0);
expect(chunk.endLine).toBeGreaterThanOrEqual(chunk.startLine);
});
});
it('should not chunk short text', () => {
const chunkText = (
text: string,
config = { maxTokens: 800, overlapPercent: 15, minChunkSize: 100 }
) => {
const lines = text.split('\n');
if (text.length < config.maxTokens * 4) {
if (text.length >= config.minChunkSize) {
return [{ content: text, startLine: 0, endLine: lines.length - 1 }];
}
return [];
}
return [{ content: text, startLine: 0, endLine: lines.length - 1 }];
};
const shortText = 'This is a short text that should not be chunked.';
const chunks = chunkText(shortText, { maxTokens: 800, overlapPercent: 15, minChunkSize: 10 });
expect(chunks.length).toBe(1);
expect(chunks[0].content).toBe(shortText);
});
});
@@ -0,0 +1,100 @@
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { QueryExpander, expandQuerySimple } from '../expansion.js';
describe('QueryExpander', () => {
let expander: QueryExpander;
beforeEach(() => {
expander = new QueryExpander();
});
afterEach(() => {
expander.clearCache();
});
describe('expandQuerySimple (without LLM)', () => {
it('should expand "auth" to authentication-related terms', () => {
const result = expandQuerySimple('auth');
expect(result.original).toBe('auth');
expect(result.variations.length).toBeGreaterThan(0);
expect(result.variations.some((v) => v.includes('authentication'))).toBe(true);
});
it('should expand "api" to related terms', () => {
const result = expandQuerySimple('api');
expect(result.original).toBe('api');
expect(result.variations.some((v) => v.includes('rest') || v.includes('endpoint'))).toBe(true);
});
it('should expand "test" to testing-related terms', () => {
const result = expandQuerySimple('testing');
expect(result.original).toBe('testing');
expect(result.variations.some((v) => v.includes('test') || v.includes('unit'))).toBe(true);
});
it('should handle queries without synonyms', () => {
const result = expandQuerySimple('xyz123unique');
expect(result.original).toBe('xyz123unique');
expect(result.variations).toEqual([]);
expect(result.weights).toEqual([2.0]);
});
it('should give original query weight of 2.0', () => {
const result = expandQuerySimple('auth');
expect(result.weights[0]).toBe(2.0);
for (let i = 1; i < result.weights.length; i++) {
expect(result.weights[i]).toBe(1.0);
}
});
it('should limit variations to 3', () => {
const result = expandQuerySimple('auth');
expect(result.variations.length).toBeLessThanOrEqual(3);
});
});
describe('expand (fallback without LLM)', () => {
it('should return expanded query without LLM initialization', async () => {
const result = await expander.expand('auth');
expect(result.original).toBe('auth');
expect(result.variations.length).toBeGreaterThan(0);
});
it('should cache results', async () => {
const result1 = await expander.expand('auth');
const result2 = await expander.expand('auth');
expect(result1).toEqual(result2);
});
it('should handle case-insensitive caching', async () => {
const result1 = await expander.expand('Auth');
const result2 = await expander.expand('AUTH');
expect(result1.original).toBe('Auth');
expect(result2.original).toBe('AUTH');
});
});
describe('clearCache', () => {
it('should clear the expansion cache', async () => {
await expander.expand('auth');
expander.clearCache();
const result = await expander.expand('auth');
expect(result).toBeDefined();
});
});
describe('isInitialized', () => {
it('should return false when not initialized with LLM', () => {
expect(expander.isInitialized()).toBe(false);
});
});
});
@@ -0,0 +1,244 @@
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import type { SkillSummary, SkillIndex } from '../../recommend/types.js';
const mockSkills: SkillSummary[] = [
{
name: 'react-auth',
description: 'Authentication utilities for React applications',
tags: ['react', 'auth', 'authentication'],
compatibility: {
frameworks: ['react'],
languages: ['typescript', 'javascript'],
libraries: ['next-auth'],
},
popularity: 500,
quality: 80,
verified: true,
},
{
name: 'vue-forms',
description: 'Form validation library for Vue',
tags: ['vue', 'forms', 'validation'],
compatibility: {
frameworks: ['vue'],
languages: ['typescript', 'javascript'],
libraries: [],
},
popularity: 300,
quality: 75,
verified: false,
},
{
name: 'api-testing',
description: 'API testing utilities for REST endpoints',
tags: ['testing', 'api', 'rest'],
compatibility: {
frameworks: [],
languages: ['typescript'],
libraries: ['jest', 'supertest'],
},
popularity: 400,
quality: 85,
verified: true,
},
{
name: 'login-helper',
description: 'Helper functions for login and authentication flows',
tags: ['auth', 'login', 'oauth'],
compatibility: {
frameworks: ['react', 'vue'],
languages: ['typescript'],
libraries: [],
},
popularity: 200,
quality: 70,
verified: false,
},
];
const mockIndex: SkillIndex = {
version: 1,
lastUpdated: new Date().toISOString(),
skills: mockSkills,
sources: [],
};
vi.mock('better-sqlite3', () => {
return {
default: vi.fn().mockImplementation(() => {
throw new Error("Cannot find module 'better-sqlite3'");
}),
};
});
vi.mock('sqlite-vec', () => {
return {
load: vi.fn(),
};
});
vi.mock('node-llama-cpp', () => {
return {
getLlama: vi.fn().mockImplementation(() => {
throw new Error("Cannot find package 'node-llama-cpp'");
}),
};
});
describe('HybridSearchPipeline', () => {
let HybridSearchPipeline: typeof import('../hybrid.js').HybridSearchPipeline;
let createHybridSearchPipeline: typeof import('../hybrid.js').createHybridSearchPipeline;
let pipeline: InstanceType<typeof HybridSearchPipeline>;
beforeEach(async () => {
const module = await import('../hybrid.js');
HybridSearchPipeline = module.HybridSearchPipeline;
createHybridSearchPipeline = module.createHybridSearchPipeline;
pipeline = createHybridSearchPipeline();
});
afterEach(async () => {
if (pipeline) {
await pipeline.dispose();
}
});
describe('initialization', () => {
it('should create a pipeline', () => {
expect(pipeline).toBeInstanceOf(HybridSearchPipeline);
});
it('should not be initialized before calling initialize()', () => {
expect(pipeline.isInitialized()).toBe(false);
});
it('should be initialized after calling initialize()', async () => {
await pipeline.initialize();
expect(pipeline.isInitialized()).toBe(true);
});
it('should load skills index', async () => {
pipeline.loadSkillsIndex(mockIndex);
const stats = pipeline.getStats();
expect(stats.skillCount).toBe(mockSkills.length);
});
});
describe('search (keyword-only fallback)', () => {
beforeEach(async () => {
pipeline.loadSkillsIndex(mockIndex);
await pipeline.initialize();
});
it('should find skills matching query', async () => {
const response = await pipeline.search({ query: 'authentication' });
expect(response.results.length).toBeGreaterThan(0);
expect(response.results.some((r) => r.skill.name === 'react-auth')).toBe(true);
});
it('should find skills by tag', async () => {
const response = await pipeline.search({ query: 'react' });
expect(response.results.length).toBeGreaterThan(0);
expect(response.results.some((r) => r.skill.tags?.includes('react'))).toBe(true);
});
it('should respect limit parameter', async () => {
const response = await pipeline.search({ query: 'auth', limit: 2 });
expect(response.results.length).toBeLessThanOrEqual(2);
});
it('should include timing information', async () => {
const response = await pipeline.search({ query: 'test' });
expect(typeof response.timing.totalMs).toBe('number');
expect(response.timing.totalMs).toBeGreaterThanOrEqual(0);
expect(response.timing.keywordSearchMs).toBeGreaterThanOrEqual(0);
});
it('should include stats', async () => {
const response = await pipeline.search({ query: 'auth' });
expect(response.stats.candidatesFromKeyword).toBeGreaterThan(0);
expect(response.stats.totalMerged).toBeGreaterThan(0);
});
it('should return original query in response', async () => {
const response = await pipeline.search({ query: 'authentication' });
expect(response.query.original).toBe('authentication');
});
});
describe('search with expansion', () => {
beforeEach(async () => {
pipeline.loadSkillsIndex(mockIndex);
await pipeline.initialize();
});
it('should expand query and find more results', async () => {
const response = await pipeline.search({
query: 'auth',
enableExpansion: true,
});
expect(response.results.length).toBeGreaterThan(0);
expect(response.query.expanded).toBeDefined();
});
it('should include expanded terms in response', async () => {
const response = await pipeline.search({
query: 'auth',
enableExpansion: true,
});
if (response.query.expanded) {
expect(response.query.expanded.original).toBe('auth');
expect(response.query.expanded.weights[0]).toBe(2.0);
}
});
});
describe('getStats', () => {
it('should return stats before initialization', () => {
const stats = pipeline.getStats();
expect(stats.initialized).toBe(false);
expect(stats.skillCount).toBe(0);
});
it('should return updated stats after loading index', async () => {
pipeline.loadSkillsIndex(mockIndex);
await pipeline.initialize();
const stats = pipeline.getStats();
expect(stats.initialized).toBe(true);
expect(stats.skillCount).toBe(mockSkills.length);
});
});
});
describe('hybridSearch helper', () => {
let hybridSearch: typeof import('../hybrid.js').hybridSearch;
beforeEach(async () => {
const module = await import('../hybrid.js');
hybridSearch = module.hybridSearch;
});
it('should perform search on skills array', async () => {
const response = await hybridSearch(mockSkills, 'react');
expect(response.results.length).toBeGreaterThan(0);
expect(response.query.original).toBe('react');
});
it('should respect options', async () => {
const response = await hybridSearch(mockSkills, 'auth', {
limit: 1,
});
expect(response.results.length).toBe(1);
});
});
@@ -0,0 +1,276 @@
import { describe, it, expect } from 'vitest';
import {
computeRRFScore,
fuseWithRRF,
normalizeScores,
weightedCombine,
applyPositionAwareBlending,
getRankFromScore,
mergeRankings,
type RankerResult,
type RRFInput,
} from '../rrf.js';
describe('RRF (Reciprocal Rank Fusion)', () => {
describe('computeRRFScore', () => {
it('should compute RRF score for a single rank', () => {
const score = computeRRFScore([1], 60);
expect(score).toBeCloseTo(1 / 61, 5);
});
it('should sum RRF scores for multiple ranks', () => {
const score = computeRRFScore([1, 2, 3], 60);
const expected = 1 / 61 + 1 / 62 + 1 / 63;
expect(score).toBeCloseTo(expected, 5);
});
it('should use custom k value', () => {
const scoreK60 = computeRRFScore([1], 60);
const scoreK30 = computeRRFScore([1], 30);
expect(scoreK30).toBeGreaterThan(scoreK60);
});
it('should return 0 for empty ranks', () => {
const score = computeRRFScore([]);
expect(score).toBe(0);
});
});
describe('fuseWithRRF', () => {
it('should fuse results from multiple rankers', () => {
const inputs: RRFInput[] = [
{
source: 'vector',
results: [
{ skillName: 'skill-a', score: 0.9 },
{ skillName: 'skill-b', score: 0.8 },
{ skillName: 'skill-c', score: 0.7 },
],
},
{
source: 'keyword',
results: [
{ skillName: 'skill-b', score: 90 },
{ skillName: 'skill-a', score: 80 },
{ skillName: 'skill-d', score: 70 },
],
},
];
const rankings = fuseWithRRF(inputs, 60);
expect(rankings.length).toBe(4);
expect(rankings[0].skillName).toBe('skill-a');
expect(rankings[1].skillName).toBe('skill-b');
});
it('should include rank information from each source', () => {
const inputs: RRFInput[] = [
{
source: 'vector',
results: [{ skillName: 'skill-a', score: 0.9 }],
},
{
source: 'keyword',
results: [{ skillName: 'skill-a', score: 90 }],
},
];
const rankings = fuseWithRRF(inputs);
expect(rankings[0].ranks).toHaveLength(2);
expect(rankings[0].ranks.map((r) => r.source)).toContain('vector');
expect(rankings[0].ranks.map((r) => r.source)).toContain('keyword');
});
it('should handle skills appearing in only one ranker', () => {
const inputs: RRFInput[] = [
{
source: 'vector',
results: [{ skillName: 'skill-a', score: 0.9 }],
},
{
source: 'keyword',
results: [{ skillName: 'skill-b', score: 90 }],
},
];
const rankings = fuseWithRRF(inputs);
expect(rankings).toHaveLength(2);
const skillAInfo = rankings.find((r) => r.skillName === 'skill-a');
const skillBInfo = rankings.find((r) => r.skillName === 'skill-b');
expect(skillAInfo?.ranks).toHaveLength(1);
expect(skillBInfo?.ranks).toHaveLength(1);
});
});
describe('normalizeScores', () => {
it('should normalize scores to 0-1 range', () => {
const results: RankerResult[] = [
{ skillName: 'a', score: 100 },
{ skillName: 'b', score: 50 },
{ skillName: 'c', score: 0 },
];
const normalized = normalizeScores(results);
expect(normalized[0].score).toBe(1);
expect(normalized[1].score).toBe(0.5);
expect(normalized[2].score).toBe(0);
});
it('should handle all same scores', () => {
const results: RankerResult[] = [
{ skillName: 'a', score: 50 },
{ skillName: 'b', score: 50 },
];
const normalized = normalizeScores(results);
expect(normalized[0].score).toBe(1);
expect(normalized[1].score).toBe(1);
});
it('should return empty array for empty input', () => {
const normalized = normalizeScores([]);
expect(normalized).toEqual([]);
});
});
describe('weightedCombine', () => {
it('should combine results with weights', () => {
const inputs: RRFInput[] = [
{
source: 'vector',
results: [
{ skillName: 'a', score: 100 },
{ skillName: 'b', score: 50 },
],
},
{
source: 'keyword',
results: [
{ skillName: 'b', score: 100 },
{ skillName: 'a', score: 50 },
],
},
];
const combined = weightedCombine(inputs, { vector: 0.6, keyword: 0.4 });
expect(combined).toHaveLength(2);
const skillA = combined.find((r) => r.skillName === 'a')!;
const skillB = combined.find((r) => r.skillName === 'b')!;
expect(skillA.score).toBeCloseTo(0.6 * 1 + 0.4 * 0, 5);
expect(skillB.score).toBeCloseTo(0.6 * 0 + 0.4 * 1, 5);
});
it('should default to weight 1 for unknown sources', () => {
const inputs: RRFInput[] = [
{
source: 'unknown',
results: [{ skillName: 'a', score: 100 }],
},
];
const combined = weightedCombine(inputs, {});
expect(combined[0].score).toBe(1);
});
});
describe('applyPositionAwareBlending', () => {
it('should apply different weights based on position', () => {
const retrievalScores = new Map([
['a', 0.9],
['b', 0.8],
['c', 0.7],
['d', 0.6],
]);
const rerankerScores = new Map([
['a', 0.5],
['b', 0.6],
['c', 0.7],
['d', 0.8],
]);
const sortedSkills = ['a', 'b', 'c', 'd'];
const blended = applyPositionAwareBlending(
retrievalScores,
rerankerScores,
sortedSkills
);
const scoreA = blended.get('a')!;
expect(scoreA).toBeCloseTo(0.9 * 0.75 + 0.5 * 0.25, 5);
const scoreD = blended.get('d')!;
expect(scoreD).toBeCloseTo(0.6 * 0.6 + 0.8 * 0.4, 5);
});
it('should handle missing scores', () => {
const retrievalScores = new Map([['a', 0.9]]);
const rerankerScores = new Map<string, number>();
const sortedSkills = ['a'];
const blended = applyPositionAwareBlending(
retrievalScores,
rerankerScores,
sortedSkills
);
expect(blended.get('a')).toBeCloseTo(0.9 * 0.75, 5);
});
});
describe('getRankFromScore', () => {
it('should return correct rank for a skill', () => {
const results: RankerResult[] = [
{ skillName: 'a', score: 90 },
{ skillName: 'b', score: 80 },
{ skillName: 'c', score: 70 },
];
expect(getRankFromScore('a', results)).toBe(1);
expect(getRankFromScore('b', results)).toBe(2);
expect(getRankFromScore('c', results)).toBe(3);
});
it('should return length+1 for missing skill', () => {
const results: RankerResult[] = [
{ skillName: 'a', score: 90 },
{ skillName: 'b', score: 80 },
];
expect(getRankFromScore('missing', results)).toBe(3);
});
});
describe('mergeRankings', () => {
it('should merge primary and secondary rankings with weights', () => {
const primary = [
{ skillName: 'a', rrfScore: 0.1, ranks: [{ source: 'primary', rank: 1 }] },
{ skillName: 'b', rrfScore: 0.05, ranks: [{ source: 'primary', rank: 2 }] },
];
const secondary = [
{ skillName: 'b', rrfScore: 0.1, ranks: [{ source: 'secondary', rank: 1 }] },
{ skillName: 'c', rrfScore: 0.05, ranks: [{ source: 'secondary', rank: 2 }] },
];
const merged = mergeRankings(primary, secondary, 0.7);
expect(merged).toHaveLength(3);
const skillA = merged.find((r) => r.skillName === 'a')!;
const skillB = merged.find((r) => r.skillName === 'b')!;
const skillC = merged.find((r) => r.skillName === 'c')!;
expect(skillA.rrfScore).toBeCloseTo(0.1 * 0.7, 5);
expect(skillB.rrfScore).toBeCloseTo(0.05 * 0.7 + 0.1 * 0.3, 5);
expect(skillC.rrfScore).toBeCloseTo(0.05 * 0.3, 5);
});
});
});
+294
View File
@@ -0,0 +1,294 @@
import type { SkillSummary } from '../recommend/types.js';
import type {
SkillEmbedding,
IndexBuildCallback,
} from './types.js';
import { DEFAULT_CHUNKING_CONFIG, MODEL_REGISTRY } from './types.js';
import { LocalModelManager } from './local-models.js';
export class EmbeddingService {
private modelManager: LocalModelManager;
private model: unknown = null;
private context: unknown = null;
private initialized = false;
private dimensions: number = 768;
constructor(modelManager?: LocalModelManager) {
this.modelManager = modelManager ?? new LocalModelManager();
}
async initialize(onProgress?: IndexBuildCallback): Promise<void> {
if (this.initialized) return;
onProgress?.({
phase: 'loading',
current: 0,
total: 1,
message: 'Loading embedding model...',
});
const modelPath = await this.modelManager.ensureEmbedModel(onProgress);
try {
// @ts-ignore - node-llama-cpp is an optional dependency
const llama = await import('node-llama-cpp');
const { getLlama } = llama;
const llamaInstance = await getLlama();
this.model = await llamaInstance.loadModel({ modelPath });
this.context = await (this.model as { createEmbeddingContext: () => Promise<unknown> }).createEmbeddingContext();
const modelInfo = MODEL_REGISTRY.embeddings[
this.modelManager.getConfig().embedModel as keyof typeof MODEL_REGISTRY.embeddings
];
this.dimensions = modelInfo?.dimensions ?? 768;
this.initialized = true;
onProgress?.({
phase: 'loading',
current: 1,
total: 1,
message: 'Embedding model loaded successfully',
});
} catch (error) {
if (
error instanceof Error &&
error.message.includes("Cannot find package 'node-llama-cpp'")
) {
throw new Error(
'node-llama-cpp is not installed. Install it with: pnpm add node-llama-cpp'
);
}
throw error;
}
}
async embed(text: string): Promise<Float32Array> {
if (!this.initialized || !this.context) {
throw new Error('EmbeddingService not initialized. Call initialize() first.');
}
const ctx = this.context as {
getEmbeddingFor(text: string): Promise<{ vector: number[] }>;
};
const result = await ctx.getEmbeddingFor(text);
return new Float32Array(result.vector);
}
async embedBatch(texts: string[]): Promise<Float32Array[]> {
const results: Float32Array[] = [];
for (const text of texts) {
results.push(await this.embed(text));
}
return results;
}
async embedSkill(skill: SkillSummary): Promise<SkillEmbedding> {
const textContent = this.buildSkillText(skill);
const vector = await this.embed(textContent);
return {
skillName: skill.name,
vector,
textContent,
generatedAt: new Date().toISOString(),
};
}
async embedSkillWithChunks(
skill: SkillSummary,
fullContent?: string
): Promise<SkillEmbedding> {
const textContent = this.buildSkillText(skill);
const mainVector = await this.embed(textContent);
const chunks = fullContent ? this.chunkText(fullContent) : undefined;
let embeddedChunks: SkillEmbedding['chunks'] | undefined;
if (chunks && chunks.length > 0) {
embeddedChunks = [];
for (const chunk of chunks) {
const chunkVector = await this.embed(chunk.content);
embeddedChunks.push({
content: chunk.content,
vector: chunkVector,
startLine: chunk.startLine,
endLine: chunk.endLine,
});
}
}
return {
skillName: skill.name,
vector: mainVector,
textContent,
chunks: embeddedChunks,
generatedAt: new Date().toISOString(),
};
}
async embedSkills(
skills: SkillSummary[],
onProgress?: IndexBuildCallback
): Promise<SkillEmbedding[]> {
await this.initialize(onProgress);
const embeddings: SkillEmbedding[] = [];
const total = skills.length;
for (let i = 0; i < skills.length; i++) {
const skill = skills[i];
onProgress?.({
phase: 'embedding',
current: i + 1,
total,
skillName: skill.name,
message: `Embedding skill ${i + 1}/${total}: ${skill.name}`,
});
const embedding = await this.embedSkill(skill);
embeddings.push(embedding);
}
onProgress?.({
phase: 'complete',
current: total,
total,
message: `Embedded ${total} skills`,
});
return embeddings;
}
private buildSkillText(skill: SkillSummary): string {
const parts: string[] = [skill.name];
if (skill.description) {
parts.push(skill.description);
}
if (skill.tags && skill.tags.length > 0) {
parts.push(skill.tags.join(' '));
}
if (skill.compatibility) {
if (skill.compatibility.frameworks?.length) {
parts.push(skill.compatibility.frameworks.join(' '));
}
if (skill.compatibility.languages?.length) {
parts.push(skill.compatibility.languages.join(' '));
}
if (skill.compatibility.libraries?.length) {
parts.push(skill.compatibility.libraries.join(' '));
}
}
return parts.join(' ').toLowerCase();
}
private chunkText(
text: string,
config = DEFAULT_CHUNKING_CONFIG
): { content: string; startLine: number; endLine: number }[] {
const lines = text.split('\n');
const chunks: { content: string; startLine: number; endLine: number }[] = [];
const avgCharsPerToken = 4;
const maxCharsPerChunk = config.maxTokens * avgCharsPerToken;
const overlapChars = Math.floor(maxCharsPerChunk * (config.overlapPercent / 100));
let currentChunk: string[] = [];
let currentChars = 0;
let startLine = 0;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
const lineChars = line.length + 1;
if (currentChars + lineChars > maxCharsPerChunk && currentChunk.length > 0) {
const content = currentChunk.join('\n');
if (content.length >= config.minChunkSize) {
chunks.push({
content,
startLine,
endLine: i - 1,
});
}
const overlapLines: string[] = [];
let overlapCharsCount = 0;
for (let j = currentChunk.length - 1; j >= 0 && overlapCharsCount < overlapChars; j--) {
overlapLines.unshift(currentChunk[j]);
overlapCharsCount += currentChunk[j].length + 1;
}
currentChunk = overlapLines;
currentChars = overlapCharsCount;
startLine = Math.max(0, i - overlapLines.length);
}
currentChunk.push(line);
currentChars += lineChars;
}
if (currentChunk.length > 0) {
const content = currentChunk.join('\n');
if (content.length >= config.minChunkSize) {
chunks.push({
content,
startLine,
endLine: lines.length - 1,
});
}
}
return chunks;
}
cosineSimilarity(a: Float32Array, b: Float32Array): number {
if (a.length !== b.length) {
throw new Error(`Vector dimensions mismatch: ${a.length} vs ${b.length}`);
}
let dotProduct = 0;
let normA = 0;
let normB = 0;
for (let i = 0; i < a.length; i++) {
dotProduct += a[i] * b[i];
normA += a[i] * a[i];
normB += b[i] * b[i];
}
const magnitude = Math.sqrt(normA) * Math.sqrt(normB);
if (magnitude === 0) return 0;
return dotProduct / magnitude;
}
getDimensions(): number {
return this.dimensions;
}
isInitialized(): boolean {
return this.initialized;
}
async dispose(): Promise<void> {
if (this.context && typeof (this.context as { dispose?: () => void }).dispose === 'function') {
(this.context as { dispose: () => void }).dispose();
}
if (this.model && typeof (this.model as { dispose?: () => void }).dispose === 'function') {
(this.model as { dispose: () => void }).dispose();
}
this.context = null;
this.model = null;
this.initialized = false;
}
}
export function createEmbeddingService(modelManager?: LocalModelManager): EmbeddingService {
return new EmbeddingService(modelManager);
}
+236
View File
@@ -0,0 +1,236 @@
import type { ExpandedQuery, IndexBuildCallback } from './types.js';
import { DEFAULT_HYBRID_CONFIG } from './types.js';
import { LocalModelManager } from './local-models.js';
interface CacheEntry {
query: ExpandedQuery;
timestamp: number;
}
const expansionCache = new Map<string, CacheEntry>();
export class QueryExpander {
private modelManager: LocalModelManager;
private model: unknown = null;
private context: unknown = null;
private initialized = false;
private cacheTtlMs: number;
constructor(modelManager?: LocalModelManager, cacheTtlMs?: number) {
this.modelManager = modelManager ?? new LocalModelManager();
this.cacheTtlMs = cacheTtlMs ?? DEFAULT_HYBRID_CONFIG.expansionCacheTtlMs;
}
async initialize(onProgress?: IndexBuildCallback): Promise<void> {
if (this.initialized) return;
onProgress?.({
phase: 'loading',
current: 0,
total: 1,
message: 'Loading LLM model for query expansion...',
});
const modelPath = await this.modelManager.ensureLlmModel(onProgress);
try {
// @ts-ignore - node-llama-cpp is an optional dependency
const llamaModule = await import('node-llama-cpp');
const { getLlama, LlamaChatSession } = llamaModule;
const llama = await getLlama();
this.model = await llama.loadModel({ modelPath });
const ctx = await (this.model as { createContext: (opts?: { contextSize?: number }) => Promise<{ getSequence: () => unknown }> }).createContext({ contextSize: 2048 });
this.context = new LlamaChatSession({ contextSequence: ctx.getSequence() as never });
this.initialized = true;
onProgress?.({
phase: 'loading',
current: 1,
total: 1,
message: 'LLM model loaded successfully',
});
} catch (error) {
if (
error instanceof Error &&
error.message.includes("Cannot find package 'node-llama-cpp'")
) {
throw new Error(
'node-llama-cpp is not installed. Install it with: pnpm add node-llama-cpp'
);
}
throw error;
}
}
async expand(query: string, maxVariations: number = 3): Promise<ExpandedQuery> {
const cached = this.getFromCache(query);
if (cached) {
return cached;
}
if (!this.initialized) {
return this.expandWithoutLLM(query);
}
try {
const variations = await this.generateVariations(query, maxVariations);
const weights = [2.0, ...variations.map(() => 1.0)];
const expanded: ExpandedQuery = {
original: query,
variations,
weights,
};
this.addToCache(query, expanded);
return expanded;
} catch {
return this.expandWithoutLLM(query);
}
}
private async generateVariations(query: string, maxVariations: number): Promise<string[]> {
const session = this.context as {
prompt(text: string): Promise<string>;
};
const prompt = `Generate ${maxVariations} alternative search queries for finding software development skills related to: "${query}"
Rules:
- Each alternative should use different terminology but capture the same intent
- Keep each alternative concise (2-5 words)
- Focus on technical terms and synonyms
- Output only the alternatives, one per line, no numbering
Example:
Query: "auth for react"
Alternatives:
authentication react
login react hooks
OAuth react integration
Query: "${query}"
Alternatives:`;
const response = await session.prompt(prompt);
const variations = response
.split('\n')
.map((line) => line.trim())
.filter((line) => line.length > 0 && line.length < 100)
.filter((line) => !line.startsWith('-') && !line.match(/^\d+\./))
.slice(0, maxVariations);
return variations;
}
private expandWithoutLLM(query: string): ExpandedQuery {
const variations: string[] = [];
const queryLower = query.toLowerCase();
const synonyms: Record<string, string[]> = {
auth: ['authentication', 'login', 'oauth', 'jwt'],
authentication: ['auth', 'login', 'oauth'],
login: ['auth', 'authentication', 'signin'],
api: ['rest', 'endpoint', 'http', 'graphql'],
rest: ['api', 'http', 'endpoint'],
db: ['database', 'sql', 'storage'],
database: ['db', 'sql', 'data storage'],
test: ['testing', 'unit test', 'spec'],
testing: ['test', 'unit test', 'e2e'],
ui: ['user interface', 'frontend', 'component'],
frontend: ['ui', 'client', 'browser'],
backend: ['server', 'api', 'service'],
form: ['input', 'validation', 'submit'],
validation: ['form validation', 'input check', 'verify'],
state: ['state management', 'store', 'data flow'],
animation: ['motion', 'transition', 'animate'],
style: ['styling', 'css', 'design'],
css: ['style', 'styling', 'tailwind'],
};
const terms = queryLower.split(/\s+/);
for (const term of terms) {
const termSynonyms = synonyms[term];
if (termSynonyms) {
for (const syn of termSynonyms.slice(0, 2)) {
const variation = query.replace(new RegExp(term, 'i'), syn);
if (variation !== query && !variations.includes(variation)) {
variations.push(variation);
}
}
}
}
const limitedVariations = variations.slice(0, 3);
const weights = [2.0, ...limitedVariations.map(() => 1.0)];
return {
original: query,
variations: limitedVariations,
weights,
};
}
private getFromCache(query: string): ExpandedQuery | null {
const entry = expansionCache.get(query.toLowerCase());
if (!entry) return null;
const now = Date.now();
if (now - entry.timestamp > this.cacheTtlMs) {
expansionCache.delete(query.toLowerCase());
return null;
}
return entry.query;
}
private addToCache(query: string, expanded: ExpandedQuery): void {
const maxCacheSize = 100;
if (expansionCache.size >= maxCacheSize) {
const oldestKey = expansionCache.keys().next().value;
if (oldestKey) {
expansionCache.delete(oldestKey);
}
}
expansionCache.set(query.toLowerCase(), {
query: expanded,
timestamp: Date.now(),
});
}
clearCache(): void {
expansionCache.clear();
}
isInitialized(): boolean {
return this.initialized;
}
async dispose(): Promise<void> {
if (this.context && typeof (this.context as { dispose?: () => void }).dispose === 'function') {
(this.context as { dispose: () => void }).dispose();
}
if (this.model && typeof (this.model as { dispose?: () => void }).dispose === 'function') {
(this.model as { dispose: () => void }).dispose();
}
this.context = null;
this.model = null;
this.initialized = false;
}
}
export function createQueryExpander(
modelManager?: LocalModelManager,
cacheTtlMs?: number
): QueryExpander {
return new QueryExpander(modelManager, cacheTtlMs);
}
export function expandQuerySimple(query: string): ExpandedQuery {
const expander = new QueryExpander();
return expander['expandWithoutLLM'](query);
}
+384
View File
@@ -0,0 +1,384 @@
import type { SkillSummary, SkillIndex } from '../recommend/types.js';
import type {
HybridSearchOptions,
HybridSearchResult,
HybridSearchResponse,
ExpandedQuery,
IndexBuildCallback,
RerankerInput,
RerankerOutput,
LocalModelConfig,
} from './types.js';
import { DEFAULT_HYBRID_CONFIG } from './types.js';
import { EmbeddingService, createEmbeddingService } from './embeddings.js';
import { VectorStore, createVectorStore } from './vector-store.js';
import { QueryExpander, createQueryExpander } from './expansion.js';
import { LocalModelManager } from './local-models.js';
import {
fuseWithRRF,
applyPositionAwareBlending,
type RankerResult,
type RRFInput,
} from './rrf.js';
import { RecommendationEngine } from '../recommend/engine.js';
export class HybridSearchPipeline {
private embeddingService: EmbeddingService;
private vectorStore: VectorStore;
private queryExpander: QueryExpander;
private modelManager: LocalModelManager;
private recommendationEngine: RecommendationEngine;
private initialized = false;
private skillsMap: Map<string, SkillSummary> = new Map();
constructor(config: Partial<LocalModelConfig> = {}) {
this.modelManager = new LocalModelManager(config);
this.embeddingService = createEmbeddingService(this.modelManager);
this.vectorStore = createVectorStore(undefined, this.embeddingService);
this.queryExpander = createQueryExpander(this.modelManager);
this.recommendationEngine = new RecommendationEngine();
}
async initialize(onProgress?: IndexBuildCallback): Promise<void> {
if (this.initialized) return;
await this.vectorStore.initialize();
this.initialized = true;
onProgress?.({
phase: 'complete',
current: 1,
total: 1,
message: 'Hybrid search pipeline initialized',
});
}
async buildIndex(
skills: SkillSummary[],
onProgress?: IndexBuildCallback
): Promise<void> {
await this.initialize(onProgress);
this.skillsMap.clear();
for (const skill of skills) {
this.skillsMap.set(skill.name, skill);
}
const index: SkillIndex = {
version: 1,
lastUpdated: new Date().toISOString(),
skills,
sources: [],
};
this.recommendationEngine.loadIndex(index);
try {
await this.embeddingService.initialize(onProgress);
const embeddings = await this.embeddingService.embedSkills(skills, onProgress);
await this.vectorStore.storeBatch(embeddings, onProgress);
onProgress?.({
phase: 'complete',
current: skills.length,
total: skills.length,
message: `Indexed ${skills.length} skills with embeddings`,
});
} catch {
// Embeddings not available - will use keyword-only search
onProgress?.({
phase: 'complete',
current: skills.length,
total: skills.length,
message: `Indexed ${skills.length} skills (keyword-only, embeddings unavailable)`,
});
}
}
async search(options: HybridSearchOptions): Promise<HybridSearchResponse> {
const startTime = Date.now();
const timing: HybridSearchResponse['timing'] = { totalMs: 0 };
const stats: HybridSearchResponse['stats'] = {
candidatesFromVector: 0,
candidatesFromKeyword: 0,
totalMerged: 0,
reranked: 0,
};
await this.initialize();
const {
query,
limit = 10,
enableExpansion = false,
enableReranking = false,
rrfK = DEFAULT_HYBRID_CONFIG.rrfK,
positionAwareBlending = true,
} = options;
let expandedQuery: ExpandedQuery | undefined;
const searchQueries: string[] = [query];
if (enableExpansion) {
try {
expandedQuery = await this.queryExpander.expand(query);
searchQueries.push(...expandedQuery.variations);
} catch {
expandedQuery = {
original: query,
variations: [],
weights: [2.0],
};
}
}
const rankerInputs: RRFInput[] = [];
const embeddingStart = Date.now();
let queryVector: Float32Array | null = null;
try {
if (this.embeddingService.isInitialized()) {
queryVector = await this.embeddingService.embed(query);
}
} catch {
}
timing.embeddingMs = Date.now() - embeddingStart;
if (queryVector) {
const vectorStart = Date.now();
const vectorResults = await this.vectorStore.search(queryVector, limit * 3);
timing.vectorSearchMs = Date.now() - vectorStart;
const vectorRankerResults: RankerResult[] = vectorResults.map((r) => ({
skillName: r.skillName,
score: r.similarity,
}));
rankerInputs.push({
source: 'vector',
results: vectorRankerResults,
});
stats.candidatesFromVector = vectorResults.length;
}
const keywordStart = Date.now();
const keywordResults: RankerResult[] = [];
for (let i = 0; i < searchQueries.length; i++) {
const searchQuery = searchQueries[i];
const weight = expandedQuery?.weights[i] ?? 1;
const results = this.recommendationEngine.search({
query: searchQuery,
limit: limit * 3,
semantic: true,
});
for (const result of results) {
const existing = keywordResults.find(
(r) => r.skillName === result.skill.name
);
if (existing) {
existing.score = Math.max(existing.score, result.relevance * weight);
} else {
keywordResults.push({
skillName: result.skill.name,
score: result.relevance * weight,
});
}
}
}
timing.keywordSearchMs = Date.now() - keywordStart;
rankerInputs.push({
source: 'keyword',
results: keywordResults,
});
stats.candidatesFromKeyword = keywordResults.length;
const fusionStart = Date.now();
const rrfRankings = fuseWithRRF(rankerInputs, rrfK);
timing.fusionMs = Date.now() - fusionStart;
stats.totalMerged = rrfRankings.length;
let finalScores = new Map<string, number>();
for (const ranking of rrfRankings) {
finalScores.set(ranking.skillName, ranking.rrfScore);
}
if (enableReranking && rrfRankings.length > 0) {
const rerankStart = Date.now();
const topCandidates = rrfRankings.slice(0, DEFAULT_HYBRID_CONFIG.rerankTopN);
const rerankerInputs: RerankerInput[] = topCandidates
.map((r) => {
const skill = this.skillsMap.get(r.skillName);
if (!skill) return null;
return {
query,
skillName: r.skillName,
skillDescription: skill.description ?? '',
skillTags: skill.tags ?? [],
};
})
.filter((r): r is RerankerInput => r !== null);
const rerankerOutputs = await this.rerank(rerankerInputs);
timing.rerankingMs = Date.now() - rerankStart;
stats.reranked = rerankerOutputs.length;
if (positionAwareBlending && rerankerOutputs.length > 0) {
const retrievalScores = new Map<string, number>();
const maxRrfScore = Math.max(...rrfRankings.map((r) => r.rrfScore));
for (const ranking of topCandidates) {
retrievalScores.set(
ranking.skillName,
ranking.rrfScore / (maxRrfScore || 1)
);
}
const rerankerScores = new Map<string, number>();
const maxRerankerScore = Math.max(...rerankerOutputs.map((r) => r.score));
for (const output of rerankerOutputs) {
rerankerScores.set(
output.skillName,
output.score / (maxRerankerScore || 1)
);
}
const sortedSkillNames = topCandidates.map((r) => r.skillName);
const blendedScores = applyPositionAwareBlending(
retrievalScores,
rerankerScores,
sortedSkillNames
);
for (const [skillName, score] of blendedScores) {
finalScores.set(skillName, score);
}
}
}
const sortedResults = Array.from(finalScores.entries())
.sort((a, b) => b[1] - a[1])
.slice(0, limit);
const vectorInput = rankerInputs.find((r) => r.source === 'vector');
const keywordInput = rankerInputs.find((r) => r.source === 'keyword');
const hybridResults: HybridSearchResult[] = [];
for (const [skillName, hybridScore] of sortedResults) {
const skill = this.skillsMap.get(skillName);
if (!skill) continue;
hybridResults.push({
skill,
relevance: Math.round(hybridScore * 100),
hybridScore,
vectorSimilarity: vectorInput?.results.find((r) => r.skillName === skillName)?.score,
keywordScore: keywordInput?.results.find((r) => r.skillName === skillName)?.score,
rrfScore: rrfRankings.find((r) => r.skillName === skillName)?.rrfScore,
matchedTerms: [],
expandedTerms: expandedQuery?.variations,
});
}
timing.totalMs = Date.now() - startTime;
return {
results: hybridResults,
query: {
original: query,
expanded: expandedQuery,
},
timing,
stats,
};
}
private async rerank(inputs: RerankerInput[]): Promise<RerankerOutput[]> {
const outputs: RerankerOutput[] = [];
for (const input of inputs) {
const baseScore =
(input.skillDescription.toLowerCase().includes(input.query.toLowerCase())
? 0.5
: 0) +
(input.skillTags.some((t) =>
t.toLowerCase().includes(input.query.toLowerCase())
)
? 0.3
: 0) +
(input.skillName.toLowerCase().includes(input.query.toLowerCase())
? 0.2
: 0);
outputs.push({
skillName: input.skillName,
score: Math.min(1, baseScore + 0.3),
});
}
return outputs;
}
loadSkillsIndex(index: SkillIndex): void {
this.skillsMap.clear();
for (const skill of index.skills) {
this.skillsMap.set(skill.name, skill);
}
this.recommendationEngine.loadIndex(index);
}
getStats(): {
vectorStore: ReturnType<VectorStore['getStats']>;
initialized: boolean;
skillCount: number;
} {
return {
vectorStore: this.vectorStore.getStats(),
initialized: this.initialized,
skillCount: this.skillsMap.size,
};
}
isInitialized(): boolean {
return this.initialized;
}
async dispose(): Promise<void> {
await this.embeddingService.dispose();
await this.queryExpander.dispose();
await this.vectorStore.close();
this.initialized = false;
}
}
export function createHybridSearchPipeline(
config?: Partial<LocalModelConfig>
): HybridSearchPipeline {
return new HybridSearchPipeline(config);
}
export async function hybridSearch(
skills: SkillSummary[],
query: string,
options: Partial<HybridSearchOptions> = {}
): Promise<HybridSearchResponse> {
const pipeline = createHybridSearchPipeline();
try {
await pipeline.buildIndex(skills);
return await pipeline.search({
query,
...options,
});
} finally {
await pipeline.dispose();
}
}
+7
View File
@@ -0,0 +1,7 @@
export * from './types.js';
export * from './local-models.js';
export * from './embeddings.js';
export * from './vector-store.js';
export * from './rrf.js';
export * from './expansion.js';
export * from './hybrid.js';
+243
View File
@@ -0,0 +1,243 @@
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
import { createWriteStream } from 'node:fs';
import type { LocalModelConfig, IndexBuildCallback } from './types.js';
import { MODEL_REGISTRY } from './types.js';
const DEFAULT_MODEL_DIR = path.join(os.homedir(), '.skillkit', 'models');
type EmbeddingModelName = keyof typeof MODEL_REGISTRY.embeddings;
type LlmModelName = keyof typeof MODEL_REGISTRY.llm;
interface ModelInfo {
url: string;
size: number;
description: string;
dimensions?: number;
}
export class LocalModelManager {
private config: Required<LocalModelConfig>;
private embedModelPath: string | null = null;
private llmModelPath: string | null = null;
constructor(config: Partial<LocalModelConfig> = {}) {
this.config = {
embedModel: config.embedModel ?? 'nomic-embed-text-v1.5.Q4_K_M.gguf',
llmModel: config.llmModel ?? 'gemma-2b-it-Q4_K_M.gguf',
modelDir: config.modelDir ?? DEFAULT_MODEL_DIR,
autoDownload: config.autoDownload ?? true,
gpuLayers: config.gpuLayers ?? 0,
};
}
async ensureModelsDirectory(): Promise<void> {
if (!fs.existsSync(this.config.modelDir)) {
await fs.promises.mkdir(this.config.modelDir, { recursive: true });
}
}
getEmbedModelPath(): string {
return path.join(this.config.modelDir, this.config.embedModel);
}
getLlmModelPath(): string {
return path.join(this.config.modelDir, this.config.llmModel);
}
isEmbedModelAvailable(): boolean {
return fs.existsSync(this.getEmbedModelPath());
}
isLlmModelAvailable(): boolean {
return fs.existsSync(this.getLlmModelPath());
}
private getModelInfo(modelType: 'embed' | 'llm', modelName: string): ModelInfo | undefined {
if (modelType === 'embed') {
return MODEL_REGISTRY.embeddings[modelName as EmbeddingModelName] as ModelInfo | undefined;
}
return MODEL_REGISTRY.llm[modelName as LlmModelName] as ModelInfo | undefined;
}
async downloadModel(
modelType: 'embed' | 'llm',
onProgress?: IndexBuildCallback
): Promise<string> {
await this.ensureModelsDirectory();
const modelName = modelType === 'embed' ? this.config.embedModel : this.config.llmModel;
const modelPath = modelType === 'embed' ? this.getEmbedModelPath() : this.getLlmModelPath();
if (fs.existsSync(modelPath)) {
return modelPath;
}
const modelInfo = this.getModelInfo(modelType, modelName);
if (!modelInfo) {
const availableModels = modelType === 'embed'
? Object.keys(MODEL_REGISTRY.embeddings)
: Object.keys(MODEL_REGISTRY.llm);
throw new Error(`Unknown model: ${modelName}. Available models: ${availableModels.join(', ')}`);
}
onProgress?.({
phase: 'downloading',
current: 0,
total: modelInfo.size,
message: `Downloading ${modelName}...`,
});
const response = await fetch(modelInfo.url);
if (!response.ok) {
throw new Error(`Failed to download model: ${response.statusText}`);
}
const contentLength = Number(response.headers.get('content-length')) || modelInfo.size;
let downloaded = 0;
const tempPath = `${modelPath}.tmp`;
const fileStream = createWriteStream(tempPath);
const reader = response.body?.getReader();
if (!reader) {
throw new Error('Failed to get response body reader');
}
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
const canContinue = fileStream.write(Buffer.from(value));
if (!canContinue) {
await new Promise<void>((resolve) => fileStream.once('drain', resolve));
}
downloaded += value.length;
onProgress?.({
phase: 'downloading',
current: downloaded,
total: contentLength,
message: `Downloading ${modelName}: ${Math.round((downloaded / contentLength) * 100)}%`,
});
}
fileStream.end();
await new Promise<void>((resolve, reject) => {
fileStream.on('finish', resolve);
fileStream.on('error', reject);
});
await fs.promises.rename(tempPath, modelPath);
return modelPath;
} catch (error) {
if (fs.existsSync(tempPath)) {
await fs.promises.unlink(tempPath);
}
throw error;
}
}
async ensureEmbedModel(onProgress?: IndexBuildCallback): Promise<string> {
if (this.embedModelPath && fs.existsSync(this.embedModelPath)) {
return this.embedModelPath;
}
if (!this.isEmbedModelAvailable()) {
if (!this.config.autoDownload) {
throw new Error(
`Embedding model not found at ${this.getEmbedModelPath()}. ` +
'Set autoDownload: true or manually download the model.'
);
}
this.embedModelPath = await this.downloadModel('embed', onProgress);
} else {
this.embedModelPath = this.getEmbedModelPath();
}
return this.embedModelPath;
}
async ensureLlmModel(onProgress?: IndexBuildCallback): Promise<string> {
if (this.llmModelPath && fs.existsSync(this.llmModelPath)) {
return this.llmModelPath;
}
if (!this.isLlmModelAvailable()) {
if (!this.config.autoDownload) {
throw new Error(
`LLM model not found at ${this.getLlmModelPath()}. ` +
'Set autoDownload: true or manually download the model.'
);
}
this.llmModelPath = await this.downloadModel('llm', onProgress);
} else {
this.llmModelPath = this.getLlmModelPath();
}
return this.llmModelPath;
}
async ensureAllModels(onProgress?: IndexBuildCallback): Promise<{
embedModel: string;
llmModel: string;
}> {
const embedModel = await this.ensureEmbedModel(onProgress);
const llmModel = await this.ensureLlmModel(onProgress);
return { embedModel, llmModel };
}
getPublicModelInfo(modelType: 'embed' | 'llm'): {
name: string;
path: string;
available: boolean;
size: number;
description: string;
} {
const modelName = modelType === 'embed' ? this.config.embedModel : this.config.llmModel;
const modelPath = modelType === 'embed' ? this.getEmbedModelPath() : this.getLlmModelPath();
const modelInfo = this.getModelInfo(modelType, modelName);
return {
name: modelName,
path: modelPath,
available: fs.existsSync(modelPath),
size: modelInfo?.size ?? 0,
description: modelInfo?.description ?? 'Unknown model',
};
}
async clearModels(): Promise<void> {
const embedPath = this.getEmbedModelPath();
const llmPath = this.getLlmModelPath();
if (fs.existsSync(embedPath)) {
await fs.promises.unlink(embedPath);
}
if (fs.existsSync(llmPath)) {
await fs.promises.unlink(llmPath);
}
this.embedModelPath = null;
this.llmModelPath = null;
}
getConfig(): Required<LocalModelConfig> {
return { ...this.config };
}
}
export function formatBytes(bytes: number): string {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const sizes = ['Bytes', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${parseFloat((bytes / Math.pow(k, i)).toFixed(2))} ${sizes[i]}`;
}
export function getDefaultModelDir(): string {
return DEFAULT_MODEL_DIR;
}
+177
View File
@@ -0,0 +1,177 @@
import type { RRFRanking } from './types.js';
import { DEFAULT_HYBRID_CONFIG } from './types.js';
export interface RankerResult {
skillName: string;
score: number;
}
export interface RRFInput {
source: string;
results: RankerResult[];
}
export function computeRRFScore(ranks: number[], k: number = DEFAULT_HYBRID_CONFIG.rrfK): number {
return ranks.reduce((sum, rank) => sum + 1 / (k + rank), 0);
}
export function fuseWithRRF(
rankerInputs: RRFInput[],
k: number = DEFAULT_HYBRID_CONFIG.rrfK
): RRFRanking[] {
const skillRanks = new Map<string, { source: string; rank: number }[]>();
for (const input of rankerInputs) {
const sortedResults = [...input.results].sort((a, b) => b.score - a.score);
for (let i = 0; i < sortedResults.length; i++) {
const result = sortedResults[i];
const rank = i + 1;
if (!skillRanks.has(result.skillName)) {
skillRanks.set(result.skillName, []);
}
skillRanks.get(result.skillName)!.push({
source: input.source,
rank,
});
}
}
const rankings: RRFRanking[] = [];
for (const [skillName, ranks] of skillRanks) {
const rankValues = ranks.map((r) => r.rank);
const rrfScore = computeRRFScore(rankValues, k);
rankings.push({
skillName,
rrfScore,
ranks,
});
}
rankings.sort((a, b) => b.rrfScore - a.rrfScore);
return rankings;
}
export function normalizeScores(results: RankerResult[]): RankerResult[] {
if (results.length === 0) return [];
const maxScore = Math.max(...results.map((r) => r.score));
const minScore = Math.min(...results.map((r) => r.score));
const range = maxScore - minScore;
if (range === 0) {
return results.map((r) => ({ ...r, score: 1 }));
}
return results.map((r) => ({
...r,
score: (r.score - minScore) / range,
}));
}
export function weightedCombine(
results: RRFInput[],
weights: Record<string, number>
): RankerResult[] {
const combinedScores = new Map<string, number>();
for (const input of results) {
const weight = weights[input.source] ?? 1;
const normalizedResults = normalizeScores(input.results);
for (const result of normalizedResults) {
const current = combinedScores.get(result.skillName) ?? 0;
combinedScores.set(result.skillName, current + result.score * weight);
}
}
return Array.from(combinedScores, ([skillName, score]) => ({ skillName, score }))
.sort((a, b) => b.score - a.score);
}
export function applyPositionAwareBlending(
retrievalScores: Map<string, number>,
rerankerScores: Map<string, number>,
sortedSkillNames: string[],
config = DEFAULT_HYBRID_CONFIG.positionBlending
): Map<string, number> {
const blendedScores = new Map<string, number>();
for (let i = 0; i < sortedSkillNames.length; i++) {
const skillName = sortedSkillNames[i];
const rank = i + 1;
let weights: { retrieval: number; reranker: number };
if (rank <= 3) {
weights = config.top3;
} else if (rank <= 10) {
weights = config.top10;
} else {
weights = config.rest;
}
const retrievalScore = retrievalScores.get(skillName) ?? 0;
const rerankerScore = rerankerScores.get(skillName) ?? 0;
const blendedScore =
retrievalScore * weights.retrieval + rerankerScore * weights.reranker;
blendedScores.set(skillName, blendedScore);
}
return blendedScores;
}
export function getRankFromScore(
skillName: string,
rankedResults: RankerResult[]
): number {
const sortedResults = [...rankedResults].sort((a, b) => b.score - a.score);
const index = sortedResults.findIndex((r) => r.skillName === skillName);
return index === -1 ? sortedResults.length + 1 : index + 1;
}
export function mergeRankings(
primaryRankings: RRFRanking[],
secondaryRankings: RRFRanking[],
primaryWeight: number = 0.7
): RRFRanking[] {
const secondaryWeight = 1 - primaryWeight;
const mergedScores = new Map<string, { score: number; ranks: { source: string; rank: number }[] }>();
for (const ranking of primaryRankings) {
mergedScores.set(ranking.skillName, {
score: ranking.rrfScore * primaryWeight,
ranks: [...ranking.ranks],
});
}
for (const ranking of secondaryRankings) {
const existing = mergedScores.get(ranking.skillName);
if (existing) {
existing.score += ranking.rrfScore * secondaryWeight;
existing.ranks = [...existing.ranks, ...ranking.ranks];
} else {
mergedScores.set(ranking.skillName, {
score: ranking.rrfScore * secondaryWeight,
ranks: [...ranking.ranks],
});
}
}
const merged: RRFRanking[] = [];
for (const [skillName, data] of mergedScores) {
merged.push({
skillName,
rrfScore: data.score,
ranks: data.ranks,
});
}
merged.sort((a, b) => b.rrfScore - a.rrfScore);
return merged;
}
+177
View File
@@ -0,0 +1,177 @@
import { z } from 'zod';
import type { SearchResult } from '../recommend/types.js';
export const LocalModelConfigSchema = z.object({
embedModel: z.string().default('nomic-embed-text-v1.5.Q4_K_M.gguf'),
llmModel: z.string().default('gemma-2b-it-Q4_K_M.gguf'),
modelDir: z.string().optional(),
autoDownload: z.boolean().default(true),
gpuLayers: z.number().default(0),
});
export type LocalModelConfig = z.infer<typeof LocalModelConfigSchema>;
export const SkillChunkSchema = z.object({
content: z.string(),
startLine: z.number(),
endLine: z.number(),
tokenCount: z.number(),
});
export type SkillChunk = z.infer<typeof SkillChunkSchema>;
export interface SkillEmbedding {
skillName: string;
vector: Float32Array;
textContent: string;
chunks?: {
content: string;
vector: Float32Array;
startLine: number;
endLine: number;
}[];
generatedAt: string;
}
export interface VectorSearchResult {
skillName: string;
similarity: number;
matchedChunk?: {
content: string;
startLine: number;
};
}
export interface ExpandedQuery {
original: string;
variations: string[];
weights: number[];
}
export interface RRFRanking {
skillName: string;
rrfScore: number;
ranks: {
source: string;
rank: number;
}[];
}
export interface HybridSearchOptions {
query: string;
limit?: number;
enableExpansion?: boolean;
enableReranking?: boolean;
semanticWeight?: number;
keywordWeight?: number;
rrfK?: number;
positionAwareBlending?: boolean;
}
export interface HybridSearchResult extends SearchResult {
hybridScore: number;
vectorSimilarity?: number;
keywordScore?: number;
rrfScore?: number;
rerankerScore?: number;
expandedTerms?: string[];
blendingWeights?: {
retrieval: number;
reranker: number;
};
}
export interface HybridSearchResponse {
results: HybridSearchResult[];
query: {
original: string;
expanded?: ExpandedQuery;
};
timing: {
totalMs: number;
embeddingMs?: number;
vectorSearchMs?: number;
keywordSearchMs?: number;
fusionMs?: number;
rerankingMs?: number;
};
stats: {
candidatesFromVector: number;
candidatesFromKeyword: number;
totalMerged: number;
reranked: number;
};
}
export interface EmbeddingServiceStats {
totalSkillsIndexed: number;
totalChunks: number;
indexSizeBytes: number;
lastIndexedAt: string;
modelName: string;
embeddingDimensions: number;
}
export interface VectorStoreConfig {
dbPath: string;
tableName?: string;
dimensions?: number;
}
export interface IndexBuildProgress {
phase: 'downloading' | 'loading' | 'embedding' | 'storing' | 'complete';
current: number;
total: number;
skillName?: string;
message?: string;
}
export type IndexBuildCallback = (progress: IndexBuildProgress) => void;
export const MODEL_REGISTRY = {
embeddings: {
'nomic-embed-text-v1.5.Q4_K_M.gguf': {
url: 'https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.Q4_K_M.gguf',
size: 547000000,
dimensions: 768,
description: 'Nomic Embed Text v1.5 - High quality embeddings',
},
},
llm: {
'gemma-2b-it-Q4_K_M.gguf': {
url: 'https://huggingface.co/bartowski/gemma-2-2b-it-GGUF/resolve/main/gemma-2-2b-it-Q4_K_M.gguf',
size: 1500000000,
description: 'Gemma 2 2B Instruct - Fast query expansion and reranking',
},
},
} as const;
export const DEFAULT_CHUNKING_CONFIG = {
maxTokens: 800,
overlapPercent: 15,
minChunkSize: 100,
} as const;
export const DEFAULT_HYBRID_CONFIG = {
rrfK: 60,
semanticWeight: 0.5,
keywordWeight: 0.5,
rerankTopN: 30,
positionBlending: {
top3: { retrieval: 0.75, reranker: 0.25 },
top10: { retrieval: 0.6, reranker: 0.4 },
rest: { retrieval: 0.4, reranker: 0.6 },
},
expansionCacheTtlMs: 5 * 60 * 1000,
} as const;
export interface RerankerInput {
query: string;
skillName: string;
skillDescription: string;
skillTags: string[];
}
export interface RerankerOutput {
skillName: string;
score: number;
reasoning?: string;
}
+510
View File
@@ -0,0 +1,510 @@
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
import type {
SkillEmbedding,
VectorSearchResult,
VectorStoreConfig,
EmbeddingServiceStats,
IndexBuildCallback,
} from './types.js';
import { EmbeddingService } from './embeddings.js';
const DEFAULT_DB_PATH = path.join(os.homedir(), '.skillkit', 'search.db');
const DEFAULT_TABLE_NAME = 'skill_embeddings';
const DEFAULT_DIMENSIONS = 768;
export class VectorStore {
private config: Required<VectorStoreConfig>;
private db: unknown = null;
private initialized = false;
private usingSqliteVec = false;
private embeddings: Map<string, SkillEmbedding> = new Map();
private embeddingService: EmbeddingService;
constructor(config: Partial<VectorStoreConfig> = {}, embeddingService?: EmbeddingService) {
const tableName = config.tableName ?? DEFAULT_TABLE_NAME;
if (!/^[a-zA-Z_][a-zA-Z0-9_]*$/.test(tableName)) {
throw new Error(`Invalid table name: ${tableName}. Must be a valid SQL identifier.`);
}
this.config = {
dbPath: config.dbPath ?? DEFAULT_DB_PATH,
tableName,
dimensions: config.dimensions ?? DEFAULT_DIMENSIONS,
};
this.embeddingService = embeddingService ?? new EmbeddingService();
}
async initialize(): Promise<void> {
if (this.initialized) return;
const dbDir = path.dirname(this.config.dbPath);
if (!fs.existsSync(dbDir)) {
await fs.promises.mkdir(dbDir, { recursive: true });
}
try {
const BetterSqlite3Module = await import('better-sqlite3');
const BetterSqlite3 = BetterSqlite3Module.default || BetterSqlite3Module;
this.db = new BetterSqlite3(this.config.dbPath);
try {
const sqliteVec = await import('sqlite-vec');
sqliteVec.load(this.db as Parameters<typeof sqliteVec.load>[0]);
this.usingSqliteVec = true;
await this.initializeSqliteVecTables();
} catch {
this.usingSqliteVec = false;
await this.initializeFallbackTables();
}
await this.loadEmbeddingsFromDb();
this.initialized = true;
} catch (error) {
if (
error instanceof Error &&
(error.message.includes("Cannot find package 'better-sqlite3'") ||
error.message.includes("Cannot find module 'better-sqlite3'"))
) {
this.initialized = true;
this.usingSqliteVec = false;
return;
}
throw error;
}
}
private async initializeSqliteVecTables(): Promise<void> {
const db = this.db as {
exec(sql: string): void;
prepare(sql: string): { run(...args: unknown[]): void };
};
db.exec(`
CREATE TABLE IF NOT EXISTS ${this.config.tableName}_meta (
skill_name TEXT PRIMARY KEY,
text_content TEXT NOT NULL,
generated_at TEXT NOT NULL,
chunk_count INTEGER DEFAULT 0
)
`);
db.exec(`
CREATE VIRTUAL TABLE IF NOT EXISTS ${this.config.tableName}_vec USING vec0(
skill_name TEXT PRIMARY KEY,
embedding float[${this.config.dimensions}]
)
`);
db.exec(`
CREATE TABLE IF NOT EXISTS ${this.config.tableName}_chunks (
id INTEGER PRIMARY KEY AUTOINCREMENT,
skill_name TEXT NOT NULL,
content TEXT NOT NULL,
start_line INTEGER NOT NULL,
end_line INTEGER NOT NULL,
FOREIGN KEY (skill_name) REFERENCES ${this.config.tableName}_meta(skill_name)
)
`);
db.exec(`
CREATE VIRTUAL TABLE IF NOT EXISTS ${this.config.tableName}_chunks_vec USING vec0(
chunk_id INTEGER PRIMARY KEY,
embedding float[${this.config.dimensions}]
)
`);
}
private async initializeFallbackTables(): Promise<void> {
const db = this.db as { exec(sql: string): void };
db.exec(`
CREATE TABLE IF NOT EXISTS ${this.config.tableName} (
skill_name TEXT PRIMARY KEY,
text_content TEXT NOT NULL,
vector BLOB NOT NULL,
generated_at TEXT NOT NULL
)
`);
db.exec(`
CREATE TABLE IF NOT EXISTS ${this.config.tableName}_chunks (
id INTEGER PRIMARY KEY AUTOINCREMENT,
skill_name TEXT NOT NULL,
content TEXT NOT NULL,
vector BLOB NOT NULL,
start_line INTEGER NOT NULL,
end_line INTEGER NOT NULL
)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_${this.config.tableName}_skill
ON ${this.config.tableName}_chunks(skill_name)
`);
}
private async loadEmbeddingsFromDb(): Promise<void> {
if (!this.db) return;
const db = this.db as {
prepare(sql: string): { all(): Array<{
skill_name: string;
text_content: string;
vector?: Buffer;
generated_at: string;
}> };
};
try {
if (this.usingSqliteVec) {
const rows = db.prepare(`
SELECT skill_name, text_content, generated_at
FROM ${this.config.tableName}_meta
`).all();
for (const row of rows) {
this.embeddings.set(row.skill_name, {
skillName: row.skill_name,
textContent: row.text_content,
vector: new Float32Array(this.config.dimensions),
generatedAt: row.generated_at,
});
}
} else {
const rows = db.prepare(`
SELECT skill_name, text_content, vector, generated_at
FROM ${this.config.tableName}
`).all();
for (const row of rows) {
if (row.vector) {
this.embeddings.set(row.skill_name, {
skillName: row.skill_name,
textContent: row.text_content,
vector: new Float32Array(row.vector.buffer),
generatedAt: row.generated_at,
});
}
}
}
} catch {
// Failed to load embeddings from DB - will start with empty in-memory store
}
}
async store(embedding: SkillEmbedding): Promise<void> {
await this.initialize();
this.embeddings.set(embedding.skillName, embedding);
if (!this.db) return;
const db = this.db as {
prepare(sql: string): {
run(...args: unknown[]): { lastInsertRowid?: number };
};
};
if (this.usingSqliteVec) {
db.prepare(`
INSERT OR REPLACE INTO ${this.config.tableName}_meta
(skill_name, text_content, generated_at, chunk_count)
VALUES (?, ?, ?, ?)
`).run(
embedding.skillName,
embedding.textContent,
embedding.generatedAt,
embedding.chunks?.length ?? 0
);
db.prepare(`
INSERT OR REPLACE INTO ${this.config.tableName}_vec
(skill_name, embedding)
VALUES (?, ?)
`).run(embedding.skillName, Buffer.from(embedding.vector.buffer));
if (embedding.chunks) {
db.prepare(`
DELETE FROM ${this.config.tableName}_chunks WHERE skill_name = ?
`).run(embedding.skillName);
for (const chunk of embedding.chunks) {
const result = db.prepare(`
INSERT INTO ${this.config.tableName}_chunks
(skill_name, content, start_line, end_line)
VALUES (?, ?, ?, ?)
`).run(embedding.skillName, chunk.content, chunk.startLine, chunk.endLine);
const rowId = result.lastInsertRowid ?? 0;
db.prepare(`
INSERT INTO ${this.config.tableName}_chunks_vec
(chunk_id, embedding)
VALUES (?, ?)
`).run(rowId, Buffer.from(chunk.vector.buffer));
}
}
} else {
db.prepare(`
INSERT OR REPLACE INTO ${this.config.tableName}
(skill_name, text_content, vector, generated_at)
VALUES (?, ?, ?, ?)
`).run(
embedding.skillName,
embedding.textContent,
Buffer.from(embedding.vector.buffer),
embedding.generatedAt
);
if (embedding.chunks) {
db.prepare(`
DELETE FROM ${this.config.tableName}_chunks WHERE skill_name = ?
`).run(embedding.skillName);
for (const chunk of embedding.chunks) {
db.prepare(`
INSERT INTO ${this.config.tableName}_chunks
(skill_name, content, vector, start_line, end_line)
VALUES (?, ?, ?, ?, ?)
`).run(
embedding.skillName,
chunk.content,
Buffer.from(chunk.vector.buffer),
chunk.startLine,
chunk.endLine
);
}
}
}
}
async storeBatch(
embeddings: SkillEmbedding[],
onProgress?: IndexBuildCallback
): Promise<void> {
await this.initialize();
const total = embeddings.length;
for (let i = 0; i < embeddings.length; i++) {
const embedding = embeddings[i];
onProgress?.({
phase: 'storing',
current: i + 1,
total,
skillName: embedding.skillName,
message: `Storing embedding ${i + 1}/${total}: ${embedding.skillName}`,
});
await this.store(embedding);
}
onProgress?.({
phase: 'complete',
current: total,
total,
message: `Stored ${total} embeddings`,
});
}
async search(
queryVector: Float32Array,
limit: number = 10
): Promise<VectorSearchResult[]> {
await this.initialize();
if (this.usingSqliteVec && this.db) {
return this.searchWithSqliteVec(queryVector, limit);
}
return this.searchInMemory(queryVector, limit);
}
private async searchWithSqliteVec(
queryVector: Float32Array,
limit: number
): Promise<VectorSearchResult[]> {
const db = this.db as {
prepare(sql: string): {
all(...args: unknown[]): Array<{ skill_name: string; distance: number }>;
};
};
const rows = db.prepare(`
SELECT skill_name, distance
FROM ${this.config.tableName}_vec
WHERE embedding MATCH ?
ORDER BY distance
LIMIT ?
`).all(Buffer.from(queryVector.buffer), limit);
return rows.map((row) => ({
skillName: row.skill_name,
similarity: 1 - row.distance,
}));
}
private searchInMemory(
queryVector: Float32Array,
limit: number
): VectorSearchResult[] {
const results: VectorSearchResult[] = [];
for (const [skillName, embedding] of this.embeddings) {
const similarity = this.embeddingService.cosineSimilarity(queryVector, embedding.vector);
results.push({ skillName, similarity });
}
results.sort((a, b) => b.similarity - a.similarity);
return results.slice(0, limit);
}
async searchChunks(
queryVector: Float32Array,
limit: number = 10
): Promise<VectorSearchResult[]> {
await this.initialize();
const results: VectorSearchResult[] = [];
for (const [skillName, embedding] of this.embeddings) {
if (!embedding.chunks || embedding.chunks.length === 0) {
const similarity = this.embeddingService.cosineSimilarity(queryVector, embedding.vector);
results.push({ skillName, similarity });
continue;
}
let bestChunk: { similarity: number; content: string; startLine: number } | null = null;
for (const chunk of embedding.chunks) {
const similarity = this.embeddingService.cosineSimilarity(queryVector, chunk.vector);
if (!bestChunk || similarity > bestChunk.similarity) {
bestChunk = {
similarity,
content: chunk.content,
startLine: chunk.startLine,
};
}
}
if (bestChunk) {
results.push({
skillName,
similarity: bestChunk.similarity,
matchedChunk: {
content: bestChunk.content,
startLine: bestChunk.startLine,
},
});
}
}
results.sort((a, b) => b.similarity - a.similarity);
return results.slice(0, limit);
}
has(skillName: string): boolean {
return this.embeddings.has(skillName);
}
get(skillName: string): SkillEmbedding | undefined {
return this.embeddings.get(skillName);
}
async delete(skillName: string): Promise<void> {
await this.initialize();
this.embeddings.delete(skillName);
if (!this.db) return;
const db = this.db as {
prepare(sql: string): { run(...args: unknown[]): void };
};
if (this.usingSqliteVec) {
db.prepare(`DELETE FROM ${this.config.tableName}_meta WHERE skill_name = ?`).run(skillName);
db.prepare(`DELETE FROM ${this.config.tableName}_vec WHERE skill_name = ?`).run(skillName);
db.prepare(`
DELETE FROM ${this.config.tableName}_chunks_vec
WHERE chunk_id IN (
SELECT id FROM ${this.config.tableName}_chunks WHERE skill_name = ?
)
`).run(skillName);
} else {
db.prepare(`DELETE FROM ${this.config.tableName} WHERE skill_name = ?`).run(skillName);
}
db.prepare(`DELETE FROM ${this.config.tableName}_chunks WHERE skill_name = ?`).run(skillName);
}
async clear(): Promise<void> {
await this.initialize();
this.embeddings.clear();
if (!this.db) return;
const db = this.db as { exec(sql: string): void };
if (this.usingSqliteVec) {
db.exec(`DELETE FROM ${this.config.tableName}_meta`);
db.exec(`DELETE FROM ${this.config.tableName}_vec`);
db.exec(`DELETE FROM ${this.config.tableName}_chunks_vec`);
} else {
db.exec(`DELETE FROM ${this.config.tableName}`);
}
db.exec(`DELETE FROM ${this.config.tableName}_chunks`);
}
getStats(): EmbeddingServiceStats {
let indexSizeBytes = 0;
let totalChunks = 0;
for (const embedding of this.embeddings.values()) {
indexSizeBytes += embedding.vector.byteLength;
indexSizeBytes += embedding.textContent.length * 2;
if (embedding.chunks) {
totalChunks += embedding.chunks.length;
for (const chunk of embedding.chunks) {
indexSizeBytes += chunk.vector.byteLength;
indexSizeBytes += chunk.content.length * 2;
}
}
}
return {
totalSkillsIndexed: this.embeddings.size,
totalChunks,
indexSizeBytes,
lastIndexedAt: new Date().toISOString(),
modelName: 'nomic-embed-text-v1.5',
embeddingDimensions: this.config.dimensions,
};
}
isInitialized(): boolean {
return this.initialized;
}
isUsingSqliteVec(): boolean {
return this.usingSqliteVec;
}
getEmbeddingCount(): number {
return this.embeddings.size;
}
async close(): Promise<void> {
if (this.db && typeof (this.db as { close?: () => void }).close === 'function') {
(this.db as { close: () => void }).close();
}
this.db = null;
this.initialized = false;
}
}
export function createVectorStore(
config?: Partial<VectorStoreConfig>,
embeddingService?: EmbeddingService
): VectorStore {
return new VectorStore(config, embeddingService);
}
+352
View File
@@ -0,0 +1,352 @@
import type { SkillSummary } from '../recommend/types.js';
import {
type TreeNode,
type SkillTree,
type TreeGeneratorOptions,
type CategoryMapping,
CATEGORY_TAXONOMY,
} from './types.js';
const DEFAULT_OPTIONS: Required<TreeGeneratorOptions> = {
maxDepth: 3,
minSkillsPerNode: 1,
includeEmpty: false,
};
export class TreeGenerator {
private options: Required<TreeGeneratorOptions>;
constructor(options?: TreeGeneratorOptions) {
this.options = { ...DEFAULT_OPTIONS, ...options };
}
generateTree(skills: SkillSummary[]): SkillTree {
const rootNode = this.buildTreeFromTaxonomy(skills);
const { totalCategories, maxDepth } = this.countTreeStats(rootNode);
return {
version: 1,
generatedAt: new Date().toISOString(),
rootNode,
totalSkills: skills.length,
totalCategories,
maxDepth,
};
}
private buildTreeFromTaxonomy(skills: SkillSummary[]): TreeNode {
const root: TreeNode = {
id: 'root',
name: 'Skills',
description: 'All available skills organized by category',
children: [],
skills: [],
skillCount: skills.length,
depth: 0,
};
const uncategorized: string[] = [];
for (const category of CATEGORY_TAXONOMY) {
const categoryNode = this.buildCategoryNode(category, skills, 1);
if (categoryNode.skillCount > 0 || this.options.includeEmpty) {
root.children.push(categoryNode);
}
}
for (const skill of skills) {
if (!this.isSkillCategorized(skill)) {
uncategorized.push(skill.name);
}
}
if (uncategorized.length > 0) {
root.children.push({
id: 'other',
name: 'Other',
description: 'Skills without specific categorization',
children: [],
skills: uncategorized,
skillCount: uncategorized.length,
depth: 1,
});
}
return root;
}
private buildCategoryNode(
category: CategoryMapping,
skills: SkillSummary[],
depth: number
): TreeNode {
const categorySkills = this.filterSkillsByCategory(skills, category);
const subcategoryNodes: TreeNode[] = [];
if (depth < this.options.maxDepth) {
for (const subcategory of category.subcategories) {
const subcategorySkills = this.filterSkillsBySubcategory(
categorySkills,
subcategory,
category
);
if (
subcategorySkills.length >= this.options.minSkillsPerNode ||
this.options.includeEmpty
) {
subcategoryNodes.push({
id: `${category.category.toLowerCase().replace(/[^a-z0-9]/g, '-')}-${subcategory.toLowerCase().replace(/[^a-z0-9]/g, '-')}`,
name: subcategory,
description: `${subcategory} skills in ${category.category}`,
children: [],
skills: subcategorySkills.map((s) => s.name),
skillCount: subcategorySkills.length,
depth: depth + 1,
});
}
}
}
const directSkills = categorySkills.filter(
(skill) =>
!subcategoryNodes.some((node) => node.skills.includes(skill.name))
);
const totalSkillCount =
directSkills.length +
subcategoryNodes.reduce((sum, node) => sum + node.skillCount, 0);
return {
id: category.category.toLowerCase().replace(/[^a-z0-9]/g, '-'),
name: category.category,
description: `${category.category} related skills`,
children: subcategoryNodes,
skills: directSkills.map((s) => s.name),
skillCount: totalSkillCount,
depth,
};
}
private filterSkillsByCategory(
skills: SkillSummary[],
category: CategoryMapping
): SkillSummary[] {
return skills.filter((skill) => {
const skillTags = (skill.tags || []).map((t) => t.toLowerCase());
const skillName = skill.name.toLowerCase();
const skillDesc = (skill.description || '').toLowerCase();
const tagMatch = skillTags.some((tag) =>
category.tags.includes(tag)
);
const keywordMatch = category.keywords.some(
(keyword) =>
skillName.includes(keyword) || skillDesc.includes(keyword)
);
const categoryNameMatch =
skillName.includes(category.category.toLowerCase()) ||
skillDesc.includes(category.category.toLowerCase());
return tagMatch || keywordMatch || categoryNameMatch;
});
}
private filterSkillsBySubcategory(
skills: SkillSummary[],
subcategory: string,
_parentCategory: CategoryMapping
): SkillSummary[] {
const subcategoryLower = subcategory.toLowerCase();
const subcategoryNormalized = subcategoryLower.replace(/[^a-z0-9]/g, '');
return skills.filter((skill) => {
const skillTags = (skill.tags || []).map((t) => t.toLowerCase());
const skillName = skill.name.toLowerCase();
const skillDesc = (skill.description || '').toLowerCase();
const tagMatch = skillTags.some(
(tag) =>
tag === subcategoryLower ||
tag === subcategoryNormalized ||
tag.includes(subcategoryLower)
);
const nameMatch =
skillName.includes(subcategoryLower) ||
skillName.includes(subcategoryNormalized);
const descMatch =
skillDesc.includes(subcategoryLower) ||
skillDesc.includes(subcategoryNormalized);
const compatibilityMatch =
skill.compatibility?.frameworks?.some(
(f) =>
f.toLowerCase() === subcategoryLower ||
f.toLowerCase() === subcategoryNormalized
) ||
skill.compatibility?.languages?.some(
(l) =>
l.toLowerCase() === subcategoryLower ||
l.toLowerCase() === subcategoryNormalized
);
return tagMatch || nameMatch || descMatch || compatibilityMatch;
});
}
private isSkillCategorized(skill: SkillSummary): boolean {
for (const category of CATEGORY_TAXONOMY) {
const skillTags = (skill.tags || []).map((t) => t.toLowerCase());
const skillName = skill.name.toLowerCase();
const skillDesc = (skill.description || '').toLowerCase();
const tagMatch = skillTags.some((tag) => category.tags.includes(tag));
const keywordMatch = category.keywords.some(
(keyword) =>
skillName.includes(keyword) || skillDesc.includes(keyword)
);
if (tagMatch || keywordMatch) {
return true;
}
}
return false;
}
private countTreeStats(node: TreeNode): {
totalCategories: number;
maxDepth: number;
} {
let totalCategories = 1;
let maxDepth = node.depth;
for (const child of node.children) {
const childStats = this.countTreeStats(child);
totalCategories += childStats.totalCategories;
maxDepth = Math.max(maxDepth, childStats.maxDepth);
}
return { totalCategories, maxDepth };
}
findNode(tree: SkillTree, path: string[]): TreeNode | null {
let current = tree.rootNode;
for (const segment of path) {
const child = current.children.find(
(c) => c.name.toLowerCase() === segment.toLowerCase()
);
if (!child) {
return null;
}
current = child;
}
return current;
}
getPath(tree: SkillTree, skillName: string): string[] | null {
const findPath = (node: TreeNode, path: string[]): string[] | null => {
if (node.skills.includes(skillName)) {
return [...path, node.name];
}
for (const child of node.children) {
const result = findPath(child, [...path, node.name]);
if (result) {
return result;
}
}
return null;
};
const result = findPath(tree.rootNode, []);
return result ? result.slice(1) : null;
}
getAllPaths(tree: SkillTree): Map<string, string[]> {
const paths = new Map<string, string[]>();
const traverse = (node: TreeNode, currentPath: string[]) => {
for (const skillName of node.skills) {
paths.set(skillName, currentPath);
}
for (const child of node.children) {
traverse(child, [...currentPath, child.name]);
}
};
traverse(tree.rootNode, []);
return paths;
}
getNodesAtDepth(tree: SkillTree, depth: number): TreeNode[] {
const nodes: TreeNode[] = [];
const traverse = (node: TreeNode) => {
if (node.depth === depth) {
nodes.push(node);
} else if (node.depth < depth) {
for (const child of node.children) {
traverse(child);
}
}
};
traverse(tree.rootNode);
return nodes;
}
flattenTree(tree: SkillTree): TreeNode[] {
const nodes: TreeNode[] = [];
const traverse = (node: TreeNode) => {
nodes.push(node);
for (const child of node.children) {
traverse(child);
}
};
traverse(tree.rootNode);
return nodes;
}
searchTree(tree: SkillTree, query: string): TreeNode[] {
const queryLower = query.toLowerCase();
const results: TreeNode[] = [];
const traverse = (node: TreeNode) => {
const nameMatch = node.name.toLowerCase().includes(queryLower);
const descMatch = (node.description || '').toLowerCase().includes(queryLower);
const skillMatch = node.skills.some((s) =>
s.toLowerCase().includes(queryLower)
);
if (nameMatch || descMatch || skillMatch) {
results.push(node);
}
for (const child of node.children) {
traverse(child);
}
};
traverse(tree.rootNode);
return results;
}
}
export function generateSkillTree(
skills: SkillSummary[],
options?: TreeGeneratorOptions
): SkillTree {
const generator = new TreeGenerator(options);
return generator.generateTree(skills);
}
+322
View File
@@ -0,0 +1,322 @@
import type { SkillSummary } from '../recommend/types.js';
import type { TreeNode, SkillTree } from './types.js';
export type RelationType = 'similar' | 'complementary' | 'dependency' | 'alternative';
export interface SkillRelation {
skillName: string;
relationType: RelationType;
strength: number;
reason: string;
}
export interface SkillNode {
name: string;
tags: string[];
source?: string;
relations: SkillRelation[];
}
export interface SkillGraph {
version: number;
generatedAt: string;
nodes: Map<string, SkillNode>;
totalSkills: number;
totalRelations: number;
}
export interface RelatedSkillResult {
skill: SkillSummary;
relationType: RelationType;
strength: number;
reason: string;
path?: string[];
}
const TAG_SIMILARITY_THRESHOLD = 0.3;
const MAX_RELATIONS_PER_SKILL = 10;
export function buildSkillGraph(skills: SkillSummary[]): SkillGraph {
const nodes = new Map<string, SkillNode>();
let totalRelations = 0;
for (const skill of skills) {
const node: SkillNode = {
name: skill.name,
tags: skill.tags || [],
source: skill.source,
relations: [],
};
nodes.set(skill.name, node);
}
for (const skill of skills) {
const node = nodes.get(skill.name);
if (!node) continue;
const relatedSkills = findRelatedSkills(skill, skills);
node.relations = relatedSkills.slice(0, MAX_RELATIONS_PER_SKILL);
totalRelations += node.relations.length;
}
return {
version: 1,
generatedAt: new Date().toISOString(),
nodes,
totalSkills: skills.length,
totalRelations,
};
}
function findRelatedSkills(
skill: SkillSummary,
allSkills: SkillSummary[]
): SkillRelation[] {
const relations: SkillRelation[] = [];
const skillTags = new Set<string>(skill.tags || []);
const skillName = skill.name.toLowerCase();
for (const other of allSkills) {
if (other.name === skill.name) continue;
const otherTags = new Set<string>(other.tags || []);
const otherName = other.name.toLowerCase();
const commonTags: string[] = [...skillTags].filter((tag) => otherTags.has(tag));
const totalTags = new Set([...skillTags, ...otherTags]).size;
const tagSimilarity = totalTags > 0 ? commonTags.length / totalTags : 0;
if (tagSimilarity >= TAG_SIMILARITY_THRESHOLD) {
const relationType = determineRelationType(skill, other, commonTags);
const reason = generateRelationReason(relationType, commonTags);
relations.push({
skillName: other.name,
relationType,
strength: Math.round(tagSimilarity * 100),
reason,
});
}
if (skillName.includes(otherName) || otherName.includes(skillName)) {
const existing = relations.find((r) => r.skillName === other.name);
if (!existing) {
relations.push({
skillName: other.name,
relationType: 'similar',
strength: 50,
reason: 'Related by name',
});
}
}
if (skill.source && other.source && skill.source === other.source) {
const existing = relations.find((r) => r.skillName === other.name);
if (existing) {
existing.strength = Math.min(100, existing.strength + 20);
} else if (commonTags.length > 0) {
relations.push({
skillName: other.name,
relationType: 'similar',
strength: 30,
reason: `Same source: ${skill.source}`,
});
}
}
}
return relations.sort((a, b) => b.strength - a.strength);
}
function determineRelationType(
skill: SkillSummary,
other: SkillSummary,
_commonTags: string[]
): RelationType {
const complementaryPairs = [
['frontend', 'backend'],
['testing', 'development'],
['security', 'api'],
['database', 'api'],
['docker', 'kubernetes'],
];
const skillTags = skill.tags || [];
const otherTags = other.tags || [];
for (const [tag1, tag2] of complementaryPairs) {
if (
(skillTags.includes(tag1) && otherTags.includes(tag2)) ||
(skillTags.includes(tag2) && otherTags.includes(tag1))
) {
return 'complementary';
}
}
const alternativeIndicators = ['vs', 'alternative', 'instead'];
const skillName = skill.name.toLowerCase();
const otherName = other.name.toLowerCase();
for (const indicator of alternativeIndicators) {
if (skillName.includes(indicator) || otherName.includes(indicator)) {
return 'alternative';
}
}
return 'similar';
}
function generateRelationReason(relationType: RelationType, commonTags: string[]): string {
switch (relationType) {
case 'similar':
return commonTags.length > 0
? `Shares tags: ${commonTags.slice(0, 3).join(', ')}`
: 'Similar functionality';
case 'complementary':
return 'Works well together';
case 'dependency':
return 'Required by this skill';
case 'alternative':
return 'Alternative approach';
default:
return 'Related skill';
}
}
export function getRelatedSkills(
skillName: string,
graph: SkillGraph,
skills: SkillSummary[],
options: {
limit?: number;
types?: RelationType[];
minStrength?: number;
} = {}
): RelatedSkillResult[] {
const { limit = 5, types, minStrength = 0 } = options;
const node = graph.nodes.get(skillName);
if (!node) return [];
const skillMap = new Map(skills.map((s) => [s.name, s]));
return node.relations
.filter((r) => {
if (types && !types.includes(r.relationType)) return false;
if (r.strength < minStrength) return false;
return true;
})
.slice(0, limit)
.map((r) => ({
skill: skillMap.get(r.skillName)!,
relationType: r.relationType,
strength: r.strength,
reason: r.reason,
}))
.filter((r) => r.skill);
}
export function findSkillsByRelationType(
graph: SkillGraph,
skills: SkillSummary[],
relationType: RelationType,
limit: number = 10
): Array<{ skill: SkillSummary; relatedCount: number }> {
const skillMap = new Map(skills.map((s) => [s.name, s]));
const counts: Map<string, number> = new Map();
for (const [_skillName, node] of graph.nodes) {
const typeRelations = node.relations.filter((r) => r.relationType === relationType);
counts.set(node.name, typeRelations.length);
}
return [...counts.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, limit)
.map(([name, count]) => ({
skill: skillMap.get(name)!,
relatedCount: count,
}))
.filter((r) => r.skill);
}
export function getSkillPath(
fromSkillName: string,
toSkillName: string,
graph: SkillGraph,
maxHops: number = 3
): string[] | null {
if (fromSkillName === toSkillName) return [fromSkillName];
const visited = new Set<string>();
const queue: Array<{ name: string; path: string[] }> = [
{ name: fromSkillName, path: [fromSkillName] },
];
while (queue.length > 0) {
const current = queue.shift()!;
if (current.path.length > maxHops + 1) continue;
const node = graph.nodes.get(current.name);
if (!node) continue;
for (const relation of node.relations) {
if (relation.skillName === toSkillName) {
return [...current.path, toSkillName];
}
if (!visited.has(relation.skillName)) {
visited.add(relation.skillName);
queue.push({
name: relation.skillName,
path: [...current.path, relation.skillName],
});
}
}
}
return null;
}
export function findSkillsInCategory(tree: SkillTree, categoryPath: string[]): string[] {
let current: TreeNode = tree.rootNode;
for (const segment of categoryPath) {
const child = current.children.find(
(c) => c.name.toLowerCase() === segment.toLowerCase()
);
if (!child) return [];
current = child;
}
return collectSkillsRecursive(current);
}
function collectSkillsRecursive(node: TreeNode): string[] {
const skills: string[] = [...node.skills];
for (const child of node.children) {
skills.push(...collectSkillsRecursive(child));
}
return skills;
}
export function serializeGraph(graph: SkillGraph): string {
const serialized = {
version: graph.version,
generatedAt: graph.generatedAt,
totalSkills: graph.totalSkills,
totalRelations: graph.totalRelations,
nodes: Object.fromEntries(graph.nodes),
};
return JSON.stringify(serialized, null, 2);
}
export function deserializeGraph(json: string): SkillGraph {
const parsed = JSON.parse(json);
return {
version: parsed.version,
generatedAt: parsed.generatedAt,
totalSkills: parsed.totalSkills,
totalRelations: parsed.totalRelations,
nodes: new Map(Object.entries(parsed.nodes)),
};
}
+25
View File
@@ -0,0 +1,25 @@
export * from './types.js';
export * from './generator.js';
export * from './serializer.js';
export * from './graph.js';
export { TreeGenerator, generateSkillTree } from './generator.js';
export {
serializeTree,
deserializeTree,
saveTree,
loadTree,
treeToText,
treeToMarkdown,
compareTreeVersions,
TREE_FILE_NAME,
} from './serializer.js';
export {
buildSkillGraph,
getRelatedSkills,
findSkillsByRelationType,
getSkillPath,
findSkillsInCategory,
serializeGraph,
deserializeGraph,
} from './graph.js';
+191
View File
@@ -0,0 +1,191 @@
import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'node:fs';
import { dirname } from 'node:path';
import type { SkillTree, TreeNode } from './types.js';
import { SkillTreeSchema } from './types.js';
export const TREE_FILE_NAME = 'skill-tree.json';
export function serializeTree(tree: SkillTree): string {
return JSON.stringify(tree, null, 2);
}
export function deserializeTree(json: string): SkillTree {
const parsed = JSON.parse(json);
return SkillTreeSchema.parse(parsed);
}
export function saveTree(tree: SkillTree, path: string): void {
const dir = dirname(path);
if (!existsSync(dir)) {
mkdirSync(dir, { recursive: true });
}
const json = serializeTree(tree);
writeFileSync(path, json, 'utf-8');
}
export function loadTree(path: string): SkillTree | null {
if (!existsSync(path)) {
return null;
}
try {
const json = readFileSync(path, 'utf-8');
return deserializeTree(json);
} catch {
return null;
}
}
export function treeToText(tree: SkillTree, options?: { maxDepth?: number }): string {
const maxDepth = options?.maxDepth ?? Infinity;
const lines: string[] = [];
lines.push(`Skill Tree (${tree.totalSkills} skills, ${tree.totalCategories} categories)`);
lines.push(`Generated: ${tree.generatedAt}`);
lines.push('');
const renderNode = (node: TreeNode, prefix: string, isLast: boolean, depth: number) => {
if (depth > maxDepth) return;
const connector = isLast ? '└── ' : '├── ';
const skillInfo =
node.skillCount > 0 ? ` (${node.skillCount} skills)` : '';
lines.push(`${prefix}${connector}${node.name}${skillInfo}`);
const newPrefix = prefix + (isLast ? ' ' : '│ ');
if (node.children.length > 0 && depth < maxDepth) {
node.children.forEach((child, index) => {
const childIsLast = index === node.children.length - 1;
renderNode(child, newPrefix, childIsLast, depth + 1);
});
}
};
const root = tree.rootNode;
lines.push(root.name);
root.children.forEach((child, index) => {
const isLast = index === root.children.length - 1;
renderNode(child, '', isLast, 1);
});
return lines.join('\n');
}
export function treeToMarkdown(tree: SkillTree): string {
const lines: string[] = [];
lines.push(`# Skill Tree`);
lines.push('');
lines.push(`> ${tree.totalSkills} skills organized in ${tree.totalCategories} categories`);
lines.push('');
lines.push(`*Generated: ${tree.generatedAt}*`);
lines.push('');
const renderNode = (node: TreeNode, depth: number) => {
const indent = ' '.repeat(depth);
const bullet = depth === 0 ? '##' : '-';
if (depth === 0) {
lines.push(`${bullet} ${node.name}`);
} else {
const skillCount = node.skillCount > 0 ? ` *(${node.skillCount})*` : '';
lines.push(`${indent}${bullet} **${node.name}**${skillCount}`);
}
if (node.skills.length > 0 && node.skills.length <= 10) {
for (const skill of node.skills) {
lines.push(`${indent} - \`${skill}\``);
}
} else if (node.skills.length > 10) {
for (const skill of node.skills.slice(0, 5)) {
lines.push(`${indent} - \`${skill}\``);
}
lines.push(`${indent} - *... and ${node.skills.length - 5} more*`);
}
for (const child of node.children) {
renderNode(child, depth + 1);
}
};
for (const category of tree.rootNode.children) {
renderNode(category, 0);
lines.push('');
}
return lines.join('\n');
}
export function compareTreeVersions(
oldTree: SkillTree,
newTree: SkillTree
): {
added: string[];
removed: string[];
moved: { skill: string; from: string[]; to: string[] }[];
} {
const oldSkills = getAllSkillsWithPaths(oldTree.rootNode);
const newSkills = getAllSkillsWithPaths(newTree.rootNode);
const oldNames = new Set(oldSkills.map((s) => s.name));
const newNames = new Set(newSkills.map((s) => s.name));
const added: string[] = [];
const removed: string[] = [];
const moved: { skill: string; from: string[]; to: string[] }[] = [];
for (const skill of newSkills) {
if (!oldNames.has(skill.name)) {
added.push(skill.name);
}
}
for (const skill of oldSkills) {
if (!newNames.has(skill.name)) {
removed.push(skill.name);
}
}
for (const newSkill of newSkills) {
const oldSkill = oldSkills.find((s) => s.name === newSkill.name);
if (oldSkill && !arraysEqual(oldSkill.path, newSkill.path)) {
moved.push({
skill: newSkill.name,
from: oldSkill.path,
to: newSkill.path,
});
}
}
return { added, removed, moved };
}
function getAllSkillsWithPaths(
node: TreeNode,
currentPath: string[] = []
): { name: string; path: string[] }[] {
const results: { name: string; path: string[] }[] = [];
const path = [...currentPath, node.name];
for (const skill of node.skills) {
results.push({ name: skill, path });
}
for (const child of node.children) {
results.push(...getAllSkillsWithPaths(child, path));
}
return results;
}
function arraysEqual(a: string[], b: string[]): boolean {
if (a.length !== b.length) return false;
for (let i = 0; i < a.length; i++) {
if (a[i] !== b[i]) return false;
}
return true;
}
+142
View File
@@ -0,0 +1,142 @@
import { z } from 'zod';
const BaseTreeNodeSchema = z.object({
id: z.string(),
name: z.string(),
description: z.string().optional(),
skills: z.array(z.string()).default([]),
skillCount: z.number().default(0),
depth: z.number().default(0),
});
export interface TreeNode {
id: string;
name: string;
description?: string;
children: TreeNode[];
skills: string[];
skillCount: number;
depth: number;
}
export const TreeNodeSchema: z.ZodType<TreeNode> = BaseTreeNodeSchema.extend({
children: z.lazy(() => z.array(TreeNodeSchema)).default([]),
}) as z.ZodType<TreeNode>;
export const SkillTreeSchema = z.object({
version: z.number().default(1),
generatedAt: z.string(),
rootNode: TreeNodeSchema,
totalSkills: z.number(),
totalCategories: z.number(),
maxDepth: z.number(),
});
export type SkillTree = z.infer<typeof SkillTreeSchema>;
export interface TreePath {
segments: string[];
node: TreeNode;
}
export interface CategoryMapping {
category: string;
subcategories: string[];
tags: string[];
keywords: string[];
}
export const CATEGORY_TAXONOMY: CategoryMapping[] = [
{
category: 'Development',
subcategories: ['Frontend', 'Backend', 'Mobile', 'Desktop', 'Full-Stack'],
tags: ['development', 'coding', 'programming', 'software'],
keywords: ['build', 'create', 'develop', 'code', 'implement'],
},
{
category: 'Frontend',
subcategories: ['React', 'Vue', 'Angular', 'Svelte', 'Solid', 'Astro', 'Next.js', 'Nuxt', 'Remix'],
tags: ['frontend', 'ui', 'ux', 'web', 'client', 'browser', 'react', 'vue', 'angular', 'svelte', 'solid', 'astro', 'nextjs', 'nuxt', 'remix'],
keywords: ['component', 'ui', 'interface', 'layout', 'style', 'css', 'html'],
},
{
category: 'Backend',
subcategories: ['Node.js', 'Python', 'Go', 'Rust', 'Java', 'C#', 'Express', 'FastAPI', 'Django', 'Flask'],
tags: ['backend', 'server', 'api', 'node', 'nodejs', 'python', 'go', 'rust', 'java', 'express', 'fastapi', 'django', 'flask', 'fastify', 'koa', 'hono', 'nestjs'],
keywords: ['server', 'api', 'endpoint', 'route', 'handler', 'middleware'],
},
{
category: 'Mobile',
subcategories: ['React Native', 'Flutter', 'iOS', 'Android', 'Expo'],
tags: ['mobile', 'ios', 'android', 'react-native', 'flutter', 'expo', 'native'],
keywords: ['app', 'mobile', 'phone', 'tablet', 'native'],
},
{
category: 'DevOps',
subcategories: ['CI/CD', 'Kubernetes', 'Docker', 'Cloud', 'Infrastructure'],
tags: ['devops', 'ci', 'cd', 'cicd', 'kubernetes', 'k8s', 'docker', 'container', 'aws', 'gcp', 'azure', 'cloud', 'infrastructure', 'iac'],
keywords: ['deploy', 'pipeline', 'container', 'orchestration', 'infra'],
},
{
category: 'Testing',
subcategories: ['Unit Testing', 'E2E Testing', 'Integration Testing', 'TDD'],
tags: ['testing', 'test', 'unit', 'e2e', 'integration', 'jest', 'vitest', 'playwright', 'cypress', 'tdd'],
keywords: ['test', 'spec', 'assertion', 'mock', 'coverage'],
},
{
category: 'Security',
subcategories: ['Authentication', 'Authorization', 'Encryption', 'Audit'],
tags: ['security', 'auth', 'authentication', 'authorization', 'oauth', 'jwt', 'encryption', 'audit', 'vulnerability'],
keywords: ['secure', 'protect', 'encrypt', 'authenticate', 'authorize'],
},
{
category: 'AI/ML',
subcategories: ['LLM Integration', 'Agents', 'RAG', 'Training', 'Inference'],
tags: ['ai', 'ml', 'llm', 'openai', 'anthropic', 'gpt', 'claude', 'agent', 'rag', 'embedding', 'vector', 'training', 'inference'],
keywords: ['ai', 'model', 'predict', 'generate', 'embed', 'chat'],
},
{
category: 'Database',
subcategories: ['SQL', 'NoSQL', 'ORM', 'Migrations'],
tags: ['database', 'db', 'sql', 'nosql', 'postgres', 'postgresql', 'mysql', 'mongodb', 'redis', 'supabase', 'firebase', 'prisma', 'drizzle'],
keywords: ['query', 'schema', 'migration', 'model', 'record'],
},
{
category: 'Tooling',
subcategories: ['Linting', 'Formatting', 'Building', 'Bundling'],
tags: ['tooling', 'tool', 'eslint', 'prettier', 'biome', 'webpack', 'vite', 'rollup', 'turbo', 'turborepo', 'monorepo'],
keywords: ['lint', 'format', 'build', 'bundle', 'compile'],
},
{
category: 'Documentation',
subcategories: ['API Docs', 'Guides', 'READMEs', 'Comments'],
tags: ['documentation', 'docs', 'readme', 'api-docs', 'jsdoc', 'typedoc', 'storybook'],
keywords: ['doc', 'document', 'guide', 'readme', 'comment'],
},
{
category: 'Performance',
subcategories: ['Optimization', 'Caching', 'Profiling', 'Monitoring'],
tags: ['performance', 'optimization', 'cache', 'caching', 'profiling', 'monitoring', 'metrics', 'observability'],
keywords: ['optimize', 'fast', 'cache', 'profile', 'monitor'],
},
];
export const TAG_TO_CATEGORY: Record<string, string[]> = {};
for (const mapping of CATEGORY_TAXONOMY) {
for (const tag of mapping.tags) {
if (!TAG_TO_CATEGORY[tag]) {
TAG_TO_CATEGORY[tag] = [];
}
TAG_TO_CATEGORY[tag].push(mapping.category);
for (const subcategory of mapping.subcategories) {
TAG_TO_CATEGORY[tag].push(`${mapping.category} > ${subcategory}`);
}
}
}
export interface TreeGeneratorOptions {
maxDepth?: number;
minSkillsPerNode?: number;
includeEmpty?: boolean;
}
+1 -1
View File
@@ -36,7 +36,7 @@ export const AgentType = z.enum([
]);
export type AgentType = z.infer<typeof AgentType>;
export const GitProvider = z.enum(['github', 'gitlab', 'bitbucket', 'local']);
export const GitProvider = z.enum(['github', 'gitlab', 'bitbucket', 'local', 'wellknown']);
export type GitProvider = z.infer<typeof GitProvider>;
export const SkillFrontmatter = z.object({
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/mcp-memory",
"version": "1.8.1",
"version": "1.10.0",
"description": "MCP (Model Context Protocol) server for SkillKit persistent memory",
"type": "module",
"main": "./dist/index.js",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/memory",
"version": "1.8.1",
"version": "1.10.0",
"description": "CozoDB-backed semantic memory with embeddings for SkillKit",
"type": "module",
"main": "./dist/index.js",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/mesh",
"version": "1.8.1",
"version": "1.10.0",
"description": "Peer mesh network for multi-machine agent distribution",
"type": "module",
"main": "./dist/index.js",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/messaging",
"version": "1.8.1",
"version": "1.10.0",
"description": "Inter-agent messaging system for SkillKit",
"type": "module",
"main": "./dist/index.js",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/resources",
"version": "1.8.1",
"version": "1.10.0",
"description": "Bundled resources for SkillKit - agents, commands, profiles, guidelines, and hooks",
"type": "module",
"main": "./dist/index.js",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@skillkit/tui",
"version": "1.8.1",
"version": "1.10.0",
"description": "Unified Terminal UI for SkillKit - Built on OpenTUI",
"type": "module",
"main": "./dist/index.js",
+313 -95
View File
@@ -11,7 +11,15 @@ import { terminalColors } from '../theme/colors.js';
import { Header } from '../components/Header.js';
import { Spinner } from '../components/Spinner.js';
import { EmptyState, ErrorState } from '../components/EmptyState.js';
import { SearchInput } from '../components/SearchInput.js';
import {
loadOrGenerateTree,
getNodeChildren,
getNodeSkills,
formatTreePath,
type TreeServiceState,
type TreeNodeDisplay,
} from '../services/index.js';
import type { TreeNode } from '@skillkit/core';
interface MarketplaceProps {
onNavigate: (screen: Screen) => void;
@@ -27,6 +35,8 @@ const CATEGORIES = [
{ name: 'AI/ML', tag: 'ai' },
];
type ViewMode = 'list' | 'tree';
export function Marketplace(props: MarketplaceProps) {
const [allSkills, setAllSkills] = createSignal<FetchedSkill[]>([]);
const [selectedIndex, setSelectedIndex] = createSignal(0);
@@ -37,6 +47,11 @@ export function Marketplace(props: MarketplaceProps) {
const [loadedRepos, setLoadedRepos] = createSignal<string[]>([]);
const [failedRepos, setFailedRepos] = createSignal<string[]>([]);
const [viewMode, setViewMode] = createSignal<ViewMode>('list');
const [treeState, setTreeState] = createSignal<TreeServiceState | null>(null);
const [currentPath, setCurrentPath] = createSignal<string[]>([]);
const [treeItems, setTreeItems] = createSignal<(TreeNodeDisplay | { type: 'skill'; name: string })[]>([]);
const rows = () => props.rows ?? 24;
createEffect(() => {
@@ -76,9 +91,32 @@ export function Marketplace(props: MarketplaceProps) {
setError(`Some repos failed (${failed.join(', ')}) and no skills loaded`);
}
try {
const tree = await loadOrGenerateTree();
setTreeState(tree);
if (tree.tree && tree.currentNode) {
updateTreeItems(tree.currentNode);
}
} catch (e) {
console.error('Failed to load skill tree:', e);
}
setLoading(false);
};
const updateTreeItems = (node: TreeNode) => {
const children = getNodeChildren(node);
const skills = getNodeSkills(node);
const items: (TreeNodeDisplay | { type: 'skill'; name: string })[] = [
...children,
...skills.map(name => ({ type: 'skill' as const, name })),
];
setTreeItems(items);
setSelectedIndex(0);
};
const filteredSkills = createMemo(() => {
let skills = allSkills();
@@ -98,9 +136,15 @@ export function Marketplace(props: MarketplaceProps) {
});
createEffect(() => {
const list = filteredSkills();
const maxIndex = Math.max(0, list.length - 1);
setSelectedIndex((prev) => Math.max(0, Math.min(prev, maxIndex)));
if (viewMode() === 'list') {
const list = filteredSkills();
const maxIndex = Math.max(0, list.length - 1);
setSelectedIndex((prev) => Math.max(0, Math.min(prev, maxIndex)));
} else {
const items = treeItems();
const maxIndex = Math.max(0, items.length - 1);
setSelectedIndex((prev) => Math.max(0, Math.min(prev, maxIndex)));
}
});
const maxVisible = () => Math.max(4, Math.floor((rows() - 14) / 2));
@@ -116,9 +160,25 @@ export function Marketplace(props: MarketplaceProps) {
return { start, items: list.slice(start, start + visible) };
});
const treeWindow = createMemo(() => {
const list = treeItems();
const selected = selectedIndex();
const visible = maxVisible();
const total = list.length;
if (total <= visible) return { start: 0, items: list };
let start = Math.max(0, selected - Math.floor(visible / 2));
start = Math.min(start, total - visible);
return { start, items: list.slice(start, start + visible) };
});
const handleKeyNav = (delta: number) => {
const max = filteredSkills().length - 1;
setSelectedIndex((prev) => Math.max(0, Math.min(prev + delta, max)));
if (viewMode() === 'list') {
const max = filteredSkills().length - 1;
setSelectedIndex((prev) => Math.max(0, Math.min(prev + delta, max)));
} else {
const max = treeItems().length - 1;
setSelectedIndex((prev) => Math.max(0, Math.min(prev + delta, max)));
}
};
const handleInstall = () => {
@@ -128,6 +188,58 @@ export function Marketplace(props: MarketplaceProps) {
}
};
const handleTreeEnter = () => {
const items = treeItems();
const item = items[selectedIndex()];
if (!item) return;
if ('type' in item && item.type === 'skill') {
props.onNavigate('installed');
} else if ('isCategory' in item && item.isCategory) {
const state = treeState();
if (state?.tree) {
const newPath = [...currentPath(), item.name];
setCurrentPath(newPath);
let node = state.tree.rootNode;
for (const segment of newPath) {
const child = node.children.find(c => c.name === segment);
if (child) {
node = child;
} else {
setCurrentPath([]);
updateTreeItems(state.tree.rootNode);
return;
}
}
updateTreeItems(node);
}
}
};
const handleTreeBack = () => {
const path = currentPath();
if (path.length === 0) {
setViewMode('list');
return;
}
const newPath = path.slice(0, -1);
setCurrentPath(newPath);
const state = treeState();
if (state?.tree) {
let node = state.tree.rootNode;
for (const segment of newPath) {
const child = node.children.find(c => c.name === segment);
if (child) {
node = child;
}
}
updateTreeItems(node);
}
};
const handleCategorySelect = (idx: number) => {
const cat = CATEGORIES[idx];
if (cat) {
@@ -136,37 +248,61 @@ export function Marketplace(props: MarketplaceProps) {
}
};
const toggleViewMode = () => {
if (viewMode() === 'list') {
setViewMode('tree');
setSelectedIndex(0);
const state = treeState();
if (state?.tree) {
updateTreeItems(state.tree.rootNode);
}
} else {
setViewMode('list');
setCurrentPath([]);
setSelectedIndex(0);
}
};
useKeyboard((key: { name?: string; sequence?: string }) => {
if (key.name === 'j' || key.name === 'down') handleKeyNav(1);
else if (key.name === 'k' || key.name === 'up') handleKeyNav(-1);
else if (key.name === 'return') handleInstall();
else if (key.name === 'return') {
if (viewMode() === 'tree') {
handleTreeEnter();
} else {
handleInstall();
}
}
else if (key.name === 'b') props.onNavigate('browse');
else if (key.name === 'r') loadData();
else if (key.sequence === 'v') toggleViewMode();
else if (key.sequence && ['1', '2', '3', '4', '5'].includes(key.sequence)) {
handleCategorySelect(parseInt(key.sequence) - 1);
if (viewMode() === 'list') {
handleCategorySelect(parseInt(key.sequence) - 1);
}
} else if (key.name === 'escape') {
if (searchQuery() || selectedCategory()) {
if (viewMode() === 'tree') {
handleTreeBack();
} else if (searchQuery() || selectedCategory()) {
setSearchQuery('');
setSelectedCategory(null);
} else {
props.onNavigate('home');
}
} else if (key.name === 'left' && viewMode() === 'tree') {
handleTreeBack();
} else if (key.name === 'right' && viewMode() === 'tree') {
handleTreeEnter();
}
});
const selectedSkill = () => {
const skills = filteredSkills();
if (skills.length === 0) return null;
return skills[selectedIndex()];
};
return (
<box flexDirection="column" paddingLeft={1}>
<Header
title="Marketplace"
subtitle="Discover and install skills"
icon="★"
count={allSkills().length}
subtitle={viewMode() === 'tree' ? `Tree View: ${formatTreePath(currentPath())}` : 'Discover and install skills'}
icon={viewMode() === 'tree' ? '🌳' : '★'}
count={viewMode() === 'tree' ? (treeState()?.tree?.totalSkills ?? 0) : allSkills().length}
/>
<Show when={error()}>
@@ -201,105 +337,187 @@ export function Marketplace(props: MarketplaceProps) {
</Show>
<text fg={terminalColors.textMuted}> | </text>
<text fg={terminalColors.text}>{allSkills().length} skills</text>
</box>
<text fg={terminalColors.textMuted}>─────────────────────────────────────────────</text>
<text> </text>
<box flexDirection="row" marginBottom={1}>
<text fg={terminalColors.text}>Categories: </text>
<For each={CATEGORIES}>
{(cat, idx) => (
<text
fg={
selectedCategory() === cat.tag
? terminalColors.accent
: terminalColors.textMuted
}
>
[{idx() + 1}]{cat.name}{' '}
</text>
)}
</For>
</box>
<text fg={terminalColors.textMuted}>─────────────────────────────────────────────</text>
<text> </text>
<Show
when={filteredSkills().length > 0}
fallback={
<EmptyState
icon="★"
title="No skills found"
description={
searchQuery() || selectedCategory()
? 'Try a different search or category'
: 'No skills loaded yet'
}
action={{ label: 'Clear Filter', key: 'Esc' }}
/>
}
>
<text fg={terminalColors.text}>
<b>
{selectedCategory() ? `${selectedCategory()} Skills` : 'Featured Skills'}
</b>{' '}
<text fg={terminalColors.textMuted}>
({filteredSkills().length} results)
</text>
<text fg={terminalColors.textMuted}> | </text>
<text fg={viewMode() === 'tree' ? terminalColors.accent : terminalColors.textMuted}>
[v] {viewMode() === 'tree' ? '🌳 Tree' : '📋 List'}
</text>
</box>
<text fg={terminalColors.textMuted}>─────────────────────────────────────────────</text>
<text> </text>
<Show when={viewMode() === 'list'}>
<box flexDirection="row" marginBottom={1}>
<text fg={terminalColors.text}>Categories: </text>
<For each={CATEGORIES}>
{(cat, idx) => (
<text
fg={
selectedCategory() === cat.tag
? terminalColors.accent
: terminalColors.textMuted
}
>
[{idx() + 1}]{cat.name}{' '}
</text>
)}
</For>
</box>
<text fg={terminalColors.textMuted}>─────────────────────────────────────────────</text>
<text> </text>
<Show when={skillsWindow().start > 0}>
<text fg={terminalColors.textMuted}> ▲ {skillsWindow().start} more</text>
<Show
when={filteredSkills().length > 0}
fallback={
<EmptyState
icon="★"
title="No skills found"
description={
searchQuery() || selectedCategory()
? 'Try a different search or category'
: 'No skills loaded yet'
}
action={{ label: 'Clear Filter', key: 'Esc' }}
/>
}
>
<text fg={terminalColors.text}>
<b>
{selectedCategory() ? `${selectedCategory()} Skills` : 'Featured Skills'}
</b>{' '}
<text fg={terminalColors.textMuted}>
({filteredSkills().length} results)
</text>
</text>
<text> </text>
<Show when={skillsWindow().start > 0}>
<text fg={terminalColors.textMuted}> ▲ {skillsWindow().start} more</text>
</Show>
<For each={skillsWindow().items}>
{(skill, idx) => {
const originalIndex = () => skillsWindow().start + idx();
const isSelected = () => originalIndex() === selectedIndex();
return (
<box flexDirection="column" marginBottom={1}>
<box flexDirection="row">
<text
fg={isSelected() ? terminalColors.accent : terminalColors.text}
width={3}
>
{isSelected() ? '▸ ' : ' '}
</text>
<text
fg={isSelected() ? terminalColors.accent : terminalColors.text}
width={25}
>
{skill.name}
</text>
<text fg={terminalColors.textMuted}>
{skill.repoName}
</text>
</box>
<Show when={skill.description}>
<text fg={terminalColors.textMuted}>
{' '}{skill.description?.slice(0, 50)}
{(skill.description?.length || 0) > 50 ? '...' : ''}
</text>
</Show>
</box>
);
}}
</For>
<Show when={skillsWindow().start + maxVisible() < filteredSkills().length}>
<text fg={terminalColors.textMuted}>
▼ {filteredSkills().length - skillsWindow().start - maxVisible()} more
</text>
</Show>
</Show>
<For each={skillsWindow().items}>
{(skill, idx) => {
const originalIndex = () => skillsWindow().start + idx();
const isSelected = () => originalIndex() === selectedIndex();
return (
<box flexDirection="column" marginBottom={1}>
<box flexDirection="row">
</Show>
<Show when={viewMode() === 'tree'}>
<box flexDirection="row" marginBottom={1}>
<text fg={terminalColors.accent}>
📁 {currentPath().length === 0 ? 'Root' : currentPath().join(' > ')}
</text>
</box>
<text fg={terminalColors.textMuted}>─────────────────────────────────────────────</text>
<text> </text>
<Show
when={treeItems().length > 0}
fallback={
<EmptyState
icon="🌳"
title="Empty category"
description="This category has no skills"
action={{ label: 'Go Back', key: 'Esc' }}
/>
}
>
<Show when={treeWindow().start > 0}>
<text fg={terminalColors.textMuted}> ▲ {treeWindow().start} more</text>
</Show>
<For each={treeWindow().items}>
{(item, idx) => {
const originalIndex = () => treeWindow().start + idx();
const isSelected = () => originalIndex() === selectedIndex();
const isCategory = () => 'isCategory' in item && item.isCategory;
const isSkill = () => 'type' in item && item.type === 'skill';
return (
<box flexDirection="row" marginBottom={1}>
<text
fg={isSelected() ? terminalColors.accent : terminalColors.text}
width={3}
>
{isSelected() ? '▸ ' : ' '}
</text>
<text
fg={isSelected() ? terminalColors.accent : terminalColors.textMuted}
width={3}
>
{isCategory() ? '📁' : isSkill() ? '📄' : '📁'}
</text>
<text
fg={isSelected() ? terminalColors.accent : terminalColors.text}
width={25}
>
{skill.name}
</text>
<text fg={terminalColors.textMuted}>
{skill.repoName}
{'name' in item ? item.name : ''}
</text>
<Show when={isCategory() && 'skillCount' in item}>
<text fg={terminalColors.textMuted}>
({(item as TreeNodeDisplay).skillCount} skills)
</text>
</Show>
</box>
<Show when={skill.description}>
<text fg={terminalColors.textMuted}>
{' '}{skill.description?.slice(0, 50)}
{(skill.description?.length || 0) > 50 ? '...' : ''}
</text>
</Show>
</box>
);
}}
</For>
);
}}
</For>
<Show when={skillsWindow().start + maxVisible() < filteredSkills().length}>
<text fg={terminalColors.textMuted}>
▼ {filteredSkills().length - skillsWindow().start - maxVisible()} more
</text>
<Show when={treeWindow().start + maxVisible() < treeItems().length}>
<text fg={terminalColors.textMuted}>
▼ {treeItems().length - treeWindow().start - maxVisible()} more
</text>
</Show>
</Show>
</Show>
<text> </text>
<text fg={terminalColors.textMuted}>─────────────────────────────────────────────</text>
<text fg={terminalColors.textMuted}>
j/k navigate Enter install 1-5 category b browse r refresh Esc back
</text>
<Show when={viewMode() === 'list'}>
<text fg={terminalColors.textMuted}>
j/k navigate Enter install 1-5 category v tree view r refresh Esc back
</text>
</Show>
<Show when={viewMode() === 'tree'}>
<text fg={terminalColors.textMuted}>
j/k ↑↓ navigate Enter/→ open ←/Esc back v list view r refresh
</text>
</Show>
</Show>
</box>
);
+11
View File
@@ -42,3 +42,14 @@ export {
getAgentFormatInfo,
translatorService,
} from './translator.service.js';
export type { TreeServiceState, TreeNodeDisplay } from './tree.service.js';
export {
loadOrGenerateTree,
navigateToPath,
getNodeChildren,
getNodeSkills,
getTreeStats,
searchInTree,
formatTreePath,
} from './tree.service.js';
@@ -6,6 +6,12 @@ export interface RecommendationDisplay {
source: string;
tags: string[];
installCommand?: string;
treePath?: string[];
explanation?: {
matchedBecause: string[];
relevantFor: string[];
confidence: 'high' | 'medium' | 'low';
};
}
export interface RecommendationOptions {
+161
View File
@@ -0,0 +1,161 @@
import { join } from 'node:path';
import { homedir } from 'node:os';
import {
generateSkillTree,
loadTree,
saveTree,
treeToText,
loadIndex,
type SkillTree,
type TreeNode,
} from '@skillkit/core';
const TREE_PATH = join(homedir(), '.skillkit', 'skill-tree.json');
export interface TreeServiceState {
tree: SkillTree | null;
currentNode: TreeNode | null;
currentPath: string[];
loading: boolean;
error: string | null;
}
export interface TreeNodeDisplay {
id: string;
name: string;
description?: string;
skillCount: number;
childCount: number;
depth: number;
isExpanded: boolean;
isCategory: boolean;
}
export async function loadOrGenerateTree(): Promise<TreeServiceState> {
try {
let tree = loadTree(TREE_PATH);
if (!tree) {
const index = loadIndex();
if (index && index.skills.length > 0) {
tree = generateSkillTree(index.skills);
saveTree(tree, TREE_PATH);
}
}
if (!tree) {
return {
tree: null,
currentNode: null,
currentPath: [],
loading: false,
error: 'No skill index found. Run "skillkit recommend --update" first.',
};
}
return {
tree,
currentNode: tree.rootNode,
currentPath: [],
loading: false,
error: null,
};
} catch (err) {
return {
tree: null,
currentNode: null,
currentPath: [],
loading: false,
error: err instanceof Error ? err.message : 'Failed to load tree',
};
}
}
export function navigateToPath(tree: SkillTree, path: string[]): TreeNode | null {
let current = tree.rootNode;
for (const segment of path) {
const child = current.children.find(
(c) => c.name.toLowerCase() === segment.toLowerCase()
);
if (!child) {
return null;
}
current = child;
}
return current;
}
export function getNodeChildren(node: TreeNode): TreeNodeDisplay[] {
return node.children.map((child) => ({
id: child.id,
name: child.name,
description: child.description,
skillCount: child.skillCount,
childCount: child.children.length,
depth: child.depth,
isExpanded: false,
isCategory: child.children.length > 0,
}));
}
export function getNodeSkills(node: TreeNode): string[] {
return node.skills;
}
export function getTreeStats(tree: SkillTree): {
totalSkills: number;
totalCategories: number;
maxDepth: number;
topCategories: { name: string; count: number }[];
} {
const topCategories = tree.rootNode.children
.sort((a, b) => b.skillCount - a.skillCount)
.slice(0, 5)
.map((c) => ({ name: c.name, count: c.skillCount }));
return {
totalSkills: tree.totalSkills,
totalCategories: tree.totalCategories,
maxDepth: tree.maxDepth,
topCategories,
};
}
export function searchInTree(tree: SkillTree, query: string): TreeNodeDisplay[] {
const results: TreeNodeDisplay[] = [];
const queryLower = query.toLowerCase();
const traverse = (node: TreeNode) => {
const nameMatch = node.name.toLowerCase().includes(queryLower);
const skillMatch = node.skills.some((s) =>
s.toLowerCase().includes(queryLower)
);
if (nameMatch || skillMatch) {
results.push({
id: node.id,
name: node.name,
description: node.description,
skillCount: node.skillCount,
childCount: node.children.length,
depth: node.depth,
isExpanded: false,
isCategory: node.children.length > 0,
});
}
for (const child of node.children) {
traverse(child);
}
};
traverse(tree.rootNode);
return results;
}
export function formatTreePath(path: string[]): string {
if (path.length === 0) return 'Root';
return path.join(' > ');
}
+1320
View File
File diff suppressed because it is too large Load Diff