Files
Rohit Ghumare b3f561dbc9 feat(quality): Quality Evaluation System Enhancements (#21)
* feat(quality): add skill quality evaluation system

- Add quality scoring module with structure, clarity, and specificity metrics
- Evaluate skills for metadata, triggers, boundaries, code examples
- Detect vague terms and penalize overly long skills
- Update validate command to include quality reports
- Add --verbose, --quality, --issues, --json flags
- Quality scores: A (90+), B (80+), C (70+), D (60+), F (<60)

* fix: address CodeRabbit and Devin review suggestions

- Fix JSON output to return exit code 1 on validation failures
- Add --min-score NaN validation with warning message
- Add directory guard for --all flag with .mdc file support
- Fix CRLF handling in frontmatter extraction
- Fix code blocks double-counting in specificity evaluation
- Fix warning message inconsistency for code examples
- Add try/catch error handling in evaluateSkillFile
- Fix isHighQuality threshold to match UI display (80)

* feat(quality): extend quality evaluation system with badges, deeper analysis, and fix command

- Phase 2: Add quality badges to CLI commands (list, recommend, install)
  - formatQualityBadge() helper for color-coded grade badges
  - --quality flag for list command
  - Low-quality skill warnings during install

- Phase 3: Add TUI quality badges
  - Extended SkillItem with quality, grade, warnings fields
  - Quality evaluation in loadSkills()
  - Grade badges in Installed and Recommend screens

- Phase 4: Deeper quality analysis
  - Deprecated pattern detection (CommonJS, class components, var)
  - Security pattern detection (hardcoded credentials, eval, innerHTML)
  - Conflicting instruction detection
  - Completeness assessment (TODOs, empty sections, example coverage)
  - New weight distribution: Structure 35%, Clarity 25%, Specificity 25%, Advanced 15%

- Phase 1: Pre-compute marketplace quality scores
  - Updated schema.json with quality field (0-100)
  - Created scripts/compute-quality.ts for batch computation

- Phase 5: Benchmark against category leaders
  - New benchmark.ts module with category detection
  - Percentile calculation and comparison notes
  - --benchmark flag for validate command

- Phase 6: Fix command for automated quality fixes
  - skillkit fix command with --dry-run and --interactive modes
  - Auto-generates frontmatter, "When to Use", "Boundaries" sections
  - --fix flag alias on validate command

* fix: address CodeRabbit and Devin review suggestions

- Fix .mdc file name extraction in fix.ts (handles both .md and .mdc)
- Add missing advanced field to test fixtures in quality.test.ts
- Fix duplicate header detection in assessCompleteness using regex.exec
- Fix ESM compatibility in compute-quality.ts using import.meta.url
- Add trailing newline to skills.json output

* fix: address remaining CodeRabbit review suggestions

- Escape regex metacharacters in vague term patterns (fix.ts)
- Improve diff algorithm using set-based comparison (fix.ts)
- Fix validateSkill receiving file path instead of directory (validate.ts)
- Check final section in completeness assessment (index.ts)
- Add error handling and empty skills guard (compute-quality.ts)

* fix: address additional review feedback

- validate.ts: Add error when --fix is passed without targets
- index.ts: Skip headers inside fenced code blocks during completeness check
- compute-quality.ts: Use nullish coalescing (??) to preserve zero quality scores
2026-01-28 14:18:42 +00:00

106 lines
2.8 KiB
JSON

{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "SkillKit Marketplace",
"description": "Schema for SkillKit marketplace skills index",
"type": "object",
"required": ["version", "skills"],
"properties": {
"version": {
"type": "integer",
"description": "Schema version number"
},
"updatedAt": {
"type": "string",
"format": "date",
"description": "Last update date (YYYY-MM-DD)"
},
"skills": {
"type": "array",
"description": "List of skills in the marketplace",
"items": {
"$ref": "#/definitions/Skill"
}
}
},
"definitions": {
"Skill": {
"type": "object",
"required": ["id", "name", "source", "tags"],
"properties": {
"id": {
"type": "string",
"pattern": "^[a-zA-Z0-9_-]+/[a-zA-Z0-9_.-]+/[a-zA-Z0-9_-]+$",
"description": "Unique identifier: owner/repo/skill-name"
},
"name": {
"type": "string",
"minLength": 1,
"maxLength": 100,
"description": "Human-readable skill name"
},
"description": {
"type": "string",
"maxLength": 500,
"description": "Brief description of the skill"
},
"source": {
"type": "string",
"pattern": "^[a-zA-Z0-9_-]+/[a-zA-Z0-9_.-]+$",
"description": "GitHub repository: owner/repo"
},
"tags": {
"type": "array",
"items": {
"type": "string",
"pattern": "^[a-z0-9-]+$"
},
"minItems": 1,
"maxItems": 10,
"description": "Tags for categorization (lowercase, hyphenated)"
},
"author": {
"type": "string",
"description": "Skill author name or GitHub username"
},
"version": {
"type": "string",
"pattern": "^\\d+\\.\\d+\\.\\d+$",
"description": "Semantic version (e.g., 1.0.0)"
},
"agents": {
"type": "array",
"items": {
"type": "string",
"enum": [
"claude-code",
"cursor",
"codex",
"gemini-cli",
"opencode",
"antigravity",
"amp",
"clawdbot",
"droid",
"github-copilot",
"goose",
"kilo",
"kiro-cli",
"roo",
"trae",
"windsurf",
"universal"
]
},
"description": "Compatible AI agents (omit for universal compatibility)"
},
"quality": {
"type": "integer",
"minimum": 0,
"maximum": 100,
"description": "Pre-computed quality score (0-100). Verified sources default to 85."
}
}
}
}
}