resk-caching
Semantic LLM Response Caching with Vector Database Integration
Cut LLM API Costs Without Cutting Quality
resk-caching is a Bun-based backend library and server that caches LLM responses using vector databases. When a new query arrives, it finds the most semantically similar cached response and returns it instead of calling your provider — dramatically reducing API costs while keeping response quality and relevance.
Key Features
- Up to 90% API cost reduction through intelligent semantic caching, with real-time cost tracking and per-provider pricing (OpenAI, Anthropic, Google…)
- Sub-5ms cached responses vs 500ms+ API calls, with intelligent cache warming strategies
- OpenAI-compatible API for zero-cost offline development — mock LLM provider included
- Scalability & availability: cache-first rate-limit relief, circuit breaker patterns, health monitoring, graceful degradation
- Built for AI chatbot backends with Redis integration
How It Works
- You maintain a database of high-quality LLM responses to common queries, stored as vector embeddings.
- When a new query arrives, the system finds the most semantically similar cached response.
- The cached response is returned instead of making a new API call — advanced selection algorithms support business logic, user preferences, and A/B testing.
Quick Start
npm install resk-caching
Get Started with resk-caching
Reduce costs and latency for your LLM applications today.
View on GitHub View on npm Documentation