AI agent memory vs. RAG vs. context windows

A context window holds what a model can use now. RAG retrieves relevant source material for a query. Agent memory carries experience and changing state across time. They overlap, but they solve different problems. Most long-running agents need all three: active context, external knowledge, and maintained history.

The short comparison#

System	Primary question	Typical source	Changes over time?
Context window	What is the model considering now?	Current prompt, recent messages, tool results	Rebuilt every turn
RAG	Which external documents answer this query?	Files, pages, databases	Usually follows source updates
Agent memory	What has this agent or user learned and decided?	Interactions, events, decisions, observations	Must be revised deliberately

Confusion begins when one layer is asked to do the others’ work.

Context windows are working memory#

The context window is the model’s immediate workspace. It contains instructions, conversation, retrieved material, and tool output. A larger window can hold more, but every token competes for attention and cost.

It is not persistent. The application must reconstruct it on the next request. It also has no independent policy for deciding which old fact remains current.

Use the context window for the task in progress: the current plan, relevant files, recent errors, and the minimum memory required to act.

RAG retrieves external knowledge#

Retrieval-augmented generation searches an external corpus and puts relevant passages into the context. It is well suited to product documentation, policies, contracts, research, and other sources that already exist as documents.

RAG answers: what do the sources say about this question?

A classic RAG system does not necessarily know that a user changed their preference, that a decision superseded an older one, or that two observations describe the same entity. Those are memory-maintenance problems.

Agent memory carries experience#

Agent memory is written by interaction and action. It may record:

• the user’s durable preferences;
• a decision and its rationale;
• what a tool call revealed;
• an unsuccessful approach;
• a relationship discovered between projects;
• a fact that later became outdated.

Memory answers: what should remain available because of what happened before?

The distinction matters because memory has a lifecycle. It needs provenance, scope, revision, and often human correction.

Where vector databases fit#

A vector database is a retrieval substrate, not a complete memory policy. It can power semantic search for RAG or memory. It does not by itself decide what to save, how to represent a contradiction, or whether an old fact should remain active.

The same is true of a knowledge graph. Graphs express relationships and time well, but still need ingestion and maintenance rules.

A combined architecture#

Consider an agent planning a deployment:

1. The context window holds the current task and recent test output.
2. RAG retrieves the cloud provider’s current documentation.
3. Memory recalls that the team previously rejected one region because of a residency requirement.
4. The agent writes the new decision back to memory with its source and rationale.

Each layer contributes information the others do not naturally carry.

How OceanDB divides the work#

OceanDB is the persistent-memory layer. Raw entries preserve observations. Pages hold maintained semantic memory. Typed links express relationships and citations. Hybrid search combines exact and semantic retrieval.

The selected memories still enter the model’s context before it reasons. Product documentation can still live in a RAG system. OceanDB does not replace the context window or every document index; it keeps the durable state that should survive them.

Which one do you need?#

• Use only context for a short, disposable task.
• Add RAG when the agent must consult a corpus larger than the prompt.
• Add memory when prior interactions or decisions should change future behavior.
• Combine all three when the agent works across sessions with both external knowledge and evolving state.

The durable question is not how much context an agent can hold. It is how the right context remains correct and available tomorrow.