resk-caching · Documentation

Semantic LLM response cache

resk-caching is a Bun service and library that stores pre-computed LLM responses with their embeddings, matches new queries by vector similarity, and serves variants — with a JWT API, optional AES-GCM encryption at rest, metrics and tracing.

• Version 1.1.3
• Bun / TypeScript
• Apache-2.0
• Requires an embedding model

Install and run

bun add resk-caching
# or as a server
git clone https://github.com/Resk-Security/resk-caching
cd resk-caching && bun install && bun run dev   # http://localhost:3000

The server is written for Bun. It uses Bun's SQLite and native Redis client, so Node is not a supported runtime for the service.

How matching works

resk-caching is a response database, not an automatic proxy cache:

1. You pre-compute embeddings for a set of canonical queries and store the answers you want served.

2. A new query arrives with its embedding; the store returns the nearest entries by cosine similarity.

3. A variant strategy (deterministic, weighted, round-robin…) picks one stored response, optionally seeded per user for stable A/B assignment.

Embeddings are supplied by the caller, so any model or dimension works as long as it is consistent. There is no built-in embedding provider and no automatic learning from live traffic.

Quick start

Store three answers to “thank you”:

curl -X POST http://localhost:3000/api/semantic/store \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $CACHE_API_KEY" \
  -d '{
    "query": "thank you",
    "query_embedding": {"vector": [0.1, 0.2, 0.3], "dimension": 3},
    "responses": [
      {"id": "r1", "text": "You are welcome!", "metadata": {"tone": "friendly"}},
      {"id": "r2", "text": "My pleasure.",    "metadata": {"tone": "formal"}},
      {"id": "r3", "text": "No problem!",     "metadata": {"tone": "casual"}}
    ],
    "variant_strategy": "weighted",
    "weights": [3, 2, 1],
    "seed": "user:123"
  }'

Look up a similar query:

curl -X POST http://localhost:3000/api/semantic/search \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $CACHE_API_KEY" \
  -d '{
    "query": "merci",
    "query_embedding": {"vector": [0.11, 0.19, 0.29], "dimension": 3},
    "limit": 2,
    "similarity_threshold": 0.8
  }'

{
  "success": true,
  "search_result": {
    "matches": [
      {
        "entry": {"query": "thank you", "responses": [...]},
        "similarity_score": 0.997,
        "selected_response": {"id": "r1", "text": "You are welcome!", "metadata": {"tone": "friendly"}}
      }
    ],
    "total_matches": 1,
    "search_time_ms": 2
  }
}

HTTP API

Cost and performance endpoints (/api/cost/*, /api/performance/*) are documented upstream for tracking savings and cache warming; check the OpenAPI document of your deployment for the enabled subset.

Configuration

Backends: memory is a per-process map with TTL; sqlite persists to resk-cache.sqlite; redis is distributed and prefixes keys with rc:. Encryption applies to values at rest and is independent of the backend.

Limitations

• Bring your own embeddings. The service never calls an embedding model; inconsistent vectors silently degrade match quality.
• Semantic false positives. A cosine threshold of 0.8 can return an answer that is wrong in context. Tune per use case and keep a bypass path.
• Not a transparent proxy. Your application must call /api/semantic/search and fall back to the provider on a miss.
• Variant logic is per entry. Weights and strategies are stored with the query; changing them means re-storing it.
• Bun-specific. SQLite and Redis use Bun APIs; expect work to run the service on Node.
• JWT is mandatory for writes. Protect JWT_SECRET and issue short-lived tokens.

Links

• GitHub · npm · project page
• Upstream documentation (installation, Redis/SQLite vector guides)