Skip to content

Memcove

Durable, queryable data memory for LLM agents — so they can work with real datasets instead of stuffing everything into their context window.

When an agent works with tabular data — a dataframe, a query result, millions of rows of logs — its only working memory is the context window: too small for real data, gone when the session ends, and unable to compute (a model asked to sum a column or join two tables in its head miscounts and invents totals).

That gap gets patched over today — pasting rows into the prompt, vector search (great for text, useless for a GROUP BY), an ephemeral code sandbox that resets between sessions, or handing the model raw SQL access to a warehouse (powerful, but a prompt-injected DELETE or cross-tenant read waiting to happen). None is durable, large-scale, correct, and agent-safe at once.

Memcove is that missing layer. It remembers datasets by name, runs joins and aggregations over data far too large to fit in a prompt — millions to billions of rows, across many tables — with plain SQL, and hands back only what's needed: a capped preview, a downloadable file, or a live Arrow stream. The data lives in a real lakehouse and never passes through the model's context; the agent only ever sees names, previews, and links. Memory is durable across turns and other agents, and every tenant is isolated.

It speaks the Model Context Protocol (MCP), so any MCP-capable agent can use it as a tool. Underneath, datasets are Apache Iceberg tables in an object store, queried through Trino — a distributed engine, which is what lets those joins and rollups scale far past anything a context window could hold.

  1. remember_dataset   store a dataframe / file / query result
  2. query_memory       explore it with read-only SQL
  3. derive_dataset     save computed tables (joins/rollups) with lineage
  4. export_dataset     hand the user a downloadable file

Two planes

Memcove separates the small control traffic an agent touches from the bulk data it never should.

  • Control plane — the MCP server. Metadata, SQL/derivation requests, capped row previews, artifact URLs, and presigned upload handles. This is everything the LLM sees.
  • Data plane — S3 (object store) + Trino/Iceberg (query engine + catalog) + a Postgres registry + an Arrow Flight streaming server. Bulk bytes move here and never round-trip through the MCP channel.

A hard invariant runs through the design: writes go through PyIceberg; reads, derivations, and exports go through Trino — and every operation is confined to the caller's private tenant namespace.

Key concepts

  • Dataset — a named table (signups, revenue_by_user). Reference it by its bare name in SQL. Datasets are private to your tenant.
  • Tenant — the isolation boundary. Each caller maps to a private Iceberg schema (t_<id>); you can only see and query your own datasets.
  • Shared reference plane — optional read-only schemas (e.g. ref_market) every tenant can query but none can write. Discover them with discover_reference_data.
  • Lineage — derive_dataset records which datasets and SQL produced a result, so you can audit provenance with inspect_dataset.

When to use Memcove

Memcove is a structured-data memory and SQL compute layer for agents. It is not a conversational-memory or vector-search product — knowing the difference saves you from reaching for the wrong tool.

Memcove is a good fit when…

  • An agent produces or receives tabular data (dataframes, query results, uploaded files) it needs to keep across turns or share with other agents.
  • You want agents to compute with SQL — joins, rollups, filters — instead of stuffing tables into the context window.
  • Data is too big for the context window, or must persist beyond a single conversation.
  • You need multi-tenant isolation for structured data across many agents or users.
  • You have heterogeneous sources that can export parquet and want one agent-safe query layer over them.
  • You value deterministic, auditable data operations — lineage, capped previews, and exportable files.

Reach for something else when…

  • You need semantic / conversational memory or vector search (embeddings, RAG). That is a different tool (mem0, Letta, a vector database). Memcove stores structured tables and runs SQL — it is not RAG memory.
  • Your data isn't tabular — raw blobs, images, long free text. Memcove is columnar and SQL-oriented.
  • You need transactional, row-level updates (OLTP). Memcove is an analytical lakehouse: agents write via create/replace/append and read via read-only SQL, not a mutable application database.
  • You need sub-millisecond key-value lookups or a cache. Trino over Iceberg is an analytical engine, not a low-latency KV store.
  • The data is tiny and transient and fits fine in the prompt — just keep it in context.

Where to go next