Memory, self-tuning caching, and retrieval for your AI agents

const stats = await cache.stats();
// {
//   llm:  { hits: 150, misses: 50, hitRate: 0.75 },
//   tool: { hits: 300, misses: 100, hitRate: 0.75 },
//   session: { reads: 1000, writes: 500 },
//   costSavedMicros: 12500000,  // $12.50 saved
//   perTool: {
//     get_weather: { hits: 200, misses: 50, hitRate: 0.8 },
//   }
// }

agent-cache also includes toolEffectiveness() which ranks your cached tools by hit rate and recommends TTL adjustments - increase, optimal, or decrease/disable - so caching stays efficient as your workload evolves.

You can also see the benefits live — we use this caching in our own BetterDB Chat.

// Agent MCP call
await mcp.callTool(
  'cache_propose_threshold_adjust',
  {
    cache_name: 'prod-semantic',
    new_threshold: 0.075,
    reasoning:
      'hit rate 28% over 7d,'
      + ' tighten threshold',
  }
)

// Pending proposal (API response)
{
  "id": "prop_01jwx3krq5",
  "status": "pending",
  "cache_name": "production-semantic",
  "new_threshold": 0.075,
  "expires_at": "2026-05-06T12:00:00Z",
  "warnings": []
}

// Dispatcher writes to Valkey:
HSET production-semantic:__config \
  threshold 0.075

// Library picks up the change
// within seconds. No restart.

The agent observes live cache metrics via MCP read tools, proposes a config change with reasoning, and a human approves it in BetterDB Monitor. The cache library polls its config key in Valkey and swaps the policy atomically - no restart, no redeploy. Config polling is live in @betterdb/[email protected] and @betterdb/[email protected]. See the full closed-loop example

Full working examples for both packages: semantic-cache example and agent-cache example. Cache intelligence requires the Feature.CACHE_INTELLIGENCE entitlement, which is part of the Pro tier.

Drop-in adapters for LangChain, LangGraph (caching and checkpointer), LlamaIndex, OpenAI, OpenAI Responses, and Anthropic — plus the Vercel AI SDK (TypeScript) and the OpenAI Agents SDK and Pydantic AI (Python). One cache library across your entire AI stack. Pick a framework to see the code and what the adapter does.

import OpenAI from 'openai';
import { hashOpenAIRequest } from '@betterdb/agent-cache/openai';

const openai = new OpenAI();
const params = {
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What is Valkey?' }],
};

const cached = await cache.llm.check(hashOpenAIRequest(params));
if (cached.hit) return cached.response;

const response = await openai.chat.completions.create(params);
await cache.llm.store(hashOpenAIRequest(params), response.choices[0].message.content);

import Valkey from 'iovalkey';
import { AgentCache } from '@betterdb/agent-cache';

const client = new Valkey({ host: 'localhost', port: 6379 });

const cache = new AgentCache({
  client,
  tierDefaults: {
    llm:     { ttl: 3600 },
    tool:    { ttl: 300 },
    session: { ttl: 1800 },
  },
});

// LLM response caching
const params = {
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What is Valkey?' }],
  temperature: 0,
};

const result = await cache.llm.check(params);
if (!result.hit) {
  const response = await callLlm(params);
  await cache.llm.store(params, response);
}

// Tool result caching
const weather = await cache.tool.check('get_weather', { city: 'Sofia' });
if (!weather.hit) {
  const data = await getWeather({ city: 'Sofia' });
  await cache.tool.store('get_weather', { city: 'Sofia' }, JSON.stringify(data));
}

// Session state
await cache.session.set('thread-1', 'last_intent', 'book_flight');
const intent = await cache.session.get('thread-1', 'last_intent');

Works on vanilla Valkey 7+, ElastiCache, Memorystore, MemoryDB, and any Redis-compatible endpoint.

Known limitations

These apply to @betterdb/agent-cache and betterdb-agent-cache.