Files
skillkit/packages/core/src/ai/context/docs-source.ts
T
Rohit Ghumare 2441f0d275 feat(ai): add smart context generation with multi-provider support (#45)
* feat(ai): add smart context generation with multi-provider support

Implements the SkillKit Smart Context Generation system:

- Multi-Provider Support: 6 LLM providers (Anthropic, OpenAI, Google, Ollama, OpenRouter, Mock) with auto-detection via environment variables
- Multi-Source Context Engine: 4 context sources (docs, codebase, skills, memory) with relevance weighting
- Skill Composition Engine: Natural language search and intelligent skill merging
- Agent Optimization: Agent-specific optimization for 10+ agents with compatibility scoring
- Security Module: Trust scoring (0-10 scale) and prompt injection detection
- Wizard Core: 6-step interactive wizard (expertise → context-sources → composition → clarification → review → install)
- CLI Integration: `skillkit generate` command with @clack/prompts

New files:
- packages/core/src/ai/providers/{types,anthropic,openai,google,ollama,openrouter,factory}.ts
- packages/core/src/ai/context/{index,docs-source,codebase-source,skills-source,memory-source}.ts
- packages/core/src/ai/composition/{index,analyzer,merger}.ts
- packages/core/src/ai/agents/{optimizer,compatibility}.ts
- packages/core/src/ai/security/{trust-score,injection-detect}.ts
- packages/core/src/ai/wizard/{types,steps,clarification,index}.ts
- packages/cli/src/commands/generate.ts
- 6 test files for new modules

* fix(ai): address Devin/CodeRabbit review feedback

- Escape regex in codebase-source.ts to prevent ReDoS attacks
- Fix division by zero in skills-source.ts when no keywords present
- Include all config overrides in ProviderFactory cache key
- Validate AI-provided indices before indexing skills arrays
- Use limited array in ollama.ts search to avoid out-of-bounds access
- Add 120s timeout with AbortController to ollama/openrouter requests
- Fix provider precedence in wizard: config.provider > options.provider
- Fix emitComplete timing to fire after install step completes
- Guard memory lookups with memoryPersonalization opt-out check

* chore(deps): update Google AI SDK to new @google/genai package

The legacy @google/generative-ai package is deprecated as of Nov 2025.
Updated to the new unified @google/genai SDK (GA since May 2025).

* fix(ai): address remaining Devin/CodeRabbit review issues

- Update AIConfig type to support all providers (google, ollama, openrouter)
- Update getAIConfig() in CLI to detect all configured providers
- Add defensive check for expertise in clarification fallback questions
- Add null check for OpenAI choices[0] response
- Validate AI-provided indices in OpenAI/Anthropic search methods
- Add 120s timeout with AbortController to OpenAI/Anthropic requests
- Guard against zero/Infinity per-source chunk allocation in ContextEngine
- Normalize trust score weights and clamp final score to [0-10]
- Fix Windows path handling in memory-source.ts (use [\\/] regex)
- Fix token-to-character ratio consistency (use 4 throughout)
- Target 95% of max tokens in truncation to stay under limit

* chore: sync docs package versions to 1.13.0 and update lockfile

- Update docs/skillkit/package.json version to 1.13.0
- Update docs/fumadocs/package.json version to 1.13.0
- Regenerate pnpm-lock.yaml with new @google/genai dependency

* fix(ai): final code review fixes for smart context generation

- Add 'mock' to AIConfig provider union type
- Show mock provider in handleProviders() when no API keys configured
- Reserve space for truncation marker in truncateContent
- Capture pre-heading content in splitIntoSections
- Add tolerant JSON parsing in anthropic parseSearchResponse
- Use Number.isInteger for index validation in openai search
- Validate all required fields in openai generateFromExample
- Guard against zero sentences in trust-score scoreClarity
- Sanitize weight values (non-negative, finite) in TrustScorer

* fix(ai): re-validate after AI optimization and validate parseGenerateResponse

- Re-truncate content after AI optimization if it exceeds agent max context
- Add JSON extraction from code fences in parseGenerateResponse
- Validate required fields (name, description, content) in parseGenerateResponse
- Clamp confidence to [0,1] range and properly type-check optional fields
2026-02-05 15:40:14 +00:00

193 lines
5.9 KiB
TypeScript

import type { ContextChunk } from '../providers/types.js';
import type { ContextSource, ContextFetchOptions } from './index.js';
interface Context7Library {
name: string;
id: string;
description?: string;
}
interface Context7DocChunk {
content: string;
title?: string;
url?: string;
}
export class DocsSource implements ContextSource {
readonly name = 'docs' as const;
readonly displayName = 'Documentation (Context7)';
private libraryCache: Map<string, Context7Library[]> = new Map();
async fetch(query: string, options: ContextFetchOptions = {}): Promise<ContextChunk[]> {
const { maxChunks = 5 } = options;
try {
const libraries = await this.resolveLibraries(query);
if (libraries.length === 0) {
return this.fallbackSearch(query, maxChunks);
}
const chunks: ContextChunk[] = [];
for (const library of libraries.slice(0, 3)) {
const docs = await this.queryDocs(library.id, query);
for (const doc of docs.slice(0, Math.ceil(maxChunks / libraries.length))) {
chunks.push({
source: 'docs',
content: this.formatDocChunk(doc, library.name),
relevance: 0.9,
metadata: {
library: library.name,
libraryId: library.id,
title: doc.title,
url: doc.url,
},
});
}
}
return chunks.slice(0, maxChunks);
} catch (error) {
console.warn('Context7 unavailable, using fallback:', error);
return this.fallbackSearch(query, maxChunks);
}
}
async isAvailable(): Promise<boolean> {
return true;
}
private async resolveLibraries(query: string): Promise<Context7Library[]> {
const cacheKey = query.toLowerCase();
if (this.libraryCache.has(cacheKey)) {
return this.libraryCache.get(cacheKey)!;
}
try {
const keywords = this.extractKeywords(query);
const libraries: Context7Library[] = [];
for (const keyword of keywords) {
const knownLib = this.getKnownLibrary(keyword);
if (knownLib) {
libraries.push(knownLib);
}
}
this.libraryCache.set(cacheKey, libraries);
return libraries;
} catch {
return [];
}
}
private async queryDocs(libraryId: string, query: string): Promise<Context7DocChunk[]> {
const fallbackDocs = this.getFallbackDocs(libraryId, query);
return fallbackDocs;
}
private getKnownLibrary(keyword: string): Context7Library | null {
const knownLibraries: Record<string, Context7Library> = {
react: { name: 'React', id: 'facebook/react' },
vue: { name: 'Vue.js', id: 'vuejs/vue' },
svelte: { name: 'Svelte', id: 'sveltejs/svelte' },
typescript: { name: 'TypeScript', id: 'microsoft/typescript' },
node: { name: 'Node.js', id: 'nodejs/node' },
nodejs: { name: 'Node.js', id: 'nodejs/node' },
vitest: { name: 'Vitest', id: 'vitest-dev/vitest' },
jest: { name: 'Jest', id: 'jestjs/jest' },
nextjs: { name: 'Next.js', id: 'vercel/next.js' },
next: { name: 'Next.js', id: 'vercel/next.js' },
express: { name: 'Express', id: 'expressjs/express' },
fastify: { name: 'Fastify', id: 'fastify/fastify' },
prisma: { name: 'Prisma', id: 'prisma/prisma' },
drizzle: { name: 'Drizzle', id: 'drizzle-team/drizzle-orm' },
tailwind: { name: 'Tailwind CSS', id: 'tailwindlabs/tailwindcss' },
tailwindcss: { name: 'Tailwind CSS', id: 'tailwindlabs/tailwindcss' },
python: { name: 'Python', id: 'python/cpython' },
django: { name: 'Django', id: 'django/django' },
flask: { name: 'Flask', id: 'pallets/flask' },
fastapi: { name: 'FastAPI', id: 'tiangolo/fastapi' },
rust: { name: 'Rust', id: 'rust-lang/rust' },
go: { name: 'Go', id: 'golang/go' },
golang: { name: 'Go', id: 'golang/go' },
};
return knownLibraries[keyword.toLowerCase()] || null;
}
private getFallbackDocs(libraryId: string, query: string): Context7DocChunk[] {
const docs: Context7DocChunk[] = [
{
title: `${libraryId} Documentation`,
content: `Documentation for ${libraryId}. Query: ${query}. For detailed information, please refer to the official documentation.`,
url: `https://github.com/${libraryId}`,
},
];
return docs;
}
private async fallbackSearch(query: string, maxChunks: number): Promise<ContextChunk[]> {
const keywords = this.extractKeywords(query);
const chunks: ContextChunk[] = [];
for (const keyword of keywords.slice(0, maxChunks)) {
const library = this.getKnownLibrary(keyword);
if (library) {
chunks.push({
source: 'docs',
content: `## ${library.name}\n\nRefer to official ${library.name} documentation for best practices and patterns related to: ${query}`,
relevance: 0.6,
metadata: {
library: library.name,
fallback: true,
},
});
}
}
if (chunks.length === 0) {
chunks.push({
source: 'docs',
content: `## General Documentation\n\nFor "${query}", consult relevant library documentation and official guides.`,
relevance: 0.4,
metadata: { fallback: true },
});
}
return chunks.slice(0, maxChunks);
}
private extractKeywords(query: string): string[] {
const words = query
.toLowerCase()
.replace(/[^a-z0-9\s]/g, ' ')
.split(/\s+/)
.filter((w) => w.length > 2);
const techKeywords = words.filter((w) => this.getKnownLibrary(w));
const otherKeywords = words.filter((w) => !this.getKnownLibrary(w));
return [...techKeywords, ...otherKeywords];
}
private formatDocChunk(doc: Context7DocChunk, libraryName: string): string {
let content = `## ${libraryName}`;
if (doc.title) {
content += `: ${doc.title}`;
}
content += '\n\n' + doc.content;
if (doc.url) {
content += `\n\nSource: ${doc.url}`;
}
return content;
}
}