Files
skillkit/packages
Rohit Ghumare 109f00cd95 feat(core): add QMD-inspired hybrid search pipeline (#40)
* feat(core): add QMD-inspired hybrid search pipeline

Implement a hybrid search system combining vector embeddings with keyword search:

- Vector embeddings using node-llama-cpp with GGUF models (nomic-embed-text)
- RRF (Reciprocal Rank Fusion) for combining multiple rankers
- Query expansion with LLM and synonym fallback
- Position-aware score blending (different weights for top3, top10, rest)
- SQLite storage with sqlite-vec extension (falls back to in-memory)
- Auto-download of local GGUF models (~2GB)

New search module:
- embeddings.ts: Vector embedding generation service
- vector-store.ts: SQLite-based vector storage
- rrf.ts: RRF fusion algorithm
- expansion.ts: Query expansion with caching
- hybrid.ts: Main hybrid search pipeline
- local-models.ts: GGUF model management

CLI integration:
- skillkit recommend --hybrid: Enable hybrid search
- skillkit recommend --expand: Enable query expansion
- skillkit recommend --rerank: Enable LLM re-ranking
- skillkit recommend --build-index: Build embedding index

Tests: 56 new tests for search module (all passing)

* fix(search): address CodeRabbit review findings

- Add missing beforeEach import in embeddings.test.ts
- Update optional dependencies to current versions
- Fix weights/variations length mismatch in expansion.ts
- Avoid mutating input arrays in rrf.ts mergeRankings
- Delete orphaned chunk vectors in vector-store.ts delete() and clear()
- Add comment for empty catch block in vector-store.ts
- Make hybridSearch helper use buildIndex for true hybrid search
- Handle graceful degradation when embeddings unavailable
- Add node-llama-cpp mock in hybrid.test.ts

* fix(search): address CodeRabbit review round 2

- Fix gemma GGUF URL to point to valid community repo (bartowski)
- Add SQL injection protection for table names in vector-store.ts
- Fix resource leak with try/finally in hybridSearch helper
- Add backpressure handling for large model downloads
- Clamp startLine to non-negative in chunk calculation
- Add runtime validation warning for --expand/--rerank without --hybrid
- Update node-llama-cpp API usage (createEmbeddingContext, createContext)
- Update pnpm-lock.yaml for new optional dependency versions

* fix(search): use @ts-ignore for optional node-llama-cpp imports

CI doesn't install optional dependencies, so @ts-ignore is needed
instead of @ts-expect-error to suppress module not found errors.
2026-02-02 23:28:08 +00:00
..
2026-02-02 16:48:43 +00:00
2026-02-02 16:48:43 +00:00
2026-02-02 16:48:43 +00:00
2026-02-02 16:48:43 +00:00
2026-02-02 16:48:43 +00:00
2026-02-02 16:48:43 +00:00
2026-02-02 16:48:43 +00:00