Skip to main content

Spring AI Memory Source Code Analysis

Large language models are stateless by nature—each call is an isolated computation with no recollection of previous interactions. Production AI applications, however, demand continuity: a customer support agent must remember the user’s name and issue across multiple turns; a code assistant needs the history of refactoring steps. Spring AI addresses this with a dedicated Memory abstraction that transparently injects conversation history into every prompt and persists new messages after the response.

This chapter dissects the source code behind Spring AI’s memory system. We will examine the core interfaces, the MemoryAdvisor, storage backends, windowing strategies, and the design patterns that keep memory decoupled from both the model and the application code.

What Is Spring AI Memory?​

Spring AI Memory is a subsystem that captures, stores, and retrieves conversation history. It solves the fundamental mismatch between stateless LLMs and stateful conversational applications.

Key design goals:

  • Separation of concerns – Memory is not embedded in the model or the client; it is a cross‑cutting concern implemented as an advisor.
  • Pluggable storage – In‑memory, JDBC, and Cassandra backends are provided; custom repositories can be added.
  • Automatic lifecycle – The framework injects history before the model call and updates the store after the response, with no application code required.
  • Context window awareness – Memory implements sliding window policies to avoid exceeding the model’s token limit.

By default, a ChatClient does not have memory. To enable it, you attach a MemoryAdvisor backed by a ChatMemory implementation.

Memory in Spring AI Architecture​

Memory is integrated into the advisor chain, sitting between the application and the model.

The MemoryAdvisor performs two critical operations:

  1. Pre‑processing – Before the prompt reaches the model, it retrieves the conversation history from ChatMemory and adds it as prior messages.
  2. Post‑processing – After the model returns a response, it extracts the new user message and the assistant’s reply, then persists them back to the store.

This design keeps the ChatClient and ChatModel completely memory‑agnostic.

Core Interfaces and Classes​

Class / InterfaceResponsibility
ChatMemoryInterface for storing and retrieving conversation messages. Defines get(String conversationId, int lastN) and add(String conversationId, Message... messages).
InMemoryChatMemoryDefault implementation backed by a ConcurrentHashMap. Suitable for development and testing.
JdbcChatMemoryJDBC‑based implementation storing messages in a relational database.
CassandraChatMemoryCassandra‑based implementation for distributed, high‑scale deployments.
MessageWindowChatMemoryDecorator that limits history to the last N messages or a token budget, preventing context window overflow.
MemoryAdvisorThe RequestResponseAdvisor that wires memory into the advisor chain. It uses ChatMemory and an optional MessageWindowChatMemory to manage history injection and pruning.
AdvisorGeneral advisor interface; MemoryAdvisor implements it.
ChatClientOrchestrator; advisors are registered via its builder.

These interfaces provide a clean separation: ChatMemory defines the storage contract, MemoryAdvisor orchestrates its usage, and MessageWindowChatMemory handles pruning policies.

Source Code Structure​

The memory subsystem resides in the package org.springframework.ai.chat.memory:

org.springframework.ai.chat.memory
├── ChatMemory.java
├── InMemoryChatMemory.java
├── MessageWindowChatMemory.java
├── advisor
│   └── MemoryAdvisor.java
└── repository
    ├── JdbcChatMemory.java
    └── CassandraChatMemory.java

Key observations:

  • The core abstraction (ChatMemory) and the default implementation are in the main package.
  • MemoryAdvisor is in a sub‑package advisor because it implements the advisor contract.
  • Persistent implementations are in a repository sub‑package, clearly separating storage concerns.
  • The MessageWindowChatMemory is a wrapper that adds windowing capability to any ChatMemory instance.

This modular structure makes the memory layer easy to extend or replace without impacting the rest of the framework.

Conversation Lifecycle​

The memory subsystem participates in a well‑defined lifecycle for every conversational turn.

  1. New conversation – The application does not need to create a conversation explicitly; a unique ID (e.g., user session ID) is passed as part of the advisor context.
  2. Memory lookup – The advisor retrieves the most recent messages (according to the window policy) from the store.
  3. Prompt augmentation – The history is prepended to the prompt as system or prior assistant/user messages, depending on the advisor configuration.
  4. LLM invocation – The model sees the full conversation context.
  5. Memory persistence – After receiving the response, the advisor extracts the current user message and the assistant’s reply, then writes them to the store.

This automatic lifecycle makes memory transparent to the application developer.

Message Storage Strategy​

Memory management revolves around three key concepts:

  • Conversation ID – A unique string (e.g., "user-123", "session-abc") that groups messages belonging to the same logical conversation. The ChatMemory interface uses this as the primary key for storage.
  • Message ordering – Messages are stored in insertion order and retrieved in the same order (FIFO). The add() method appends; get() returns the latest N messages.
  • Window management – Unbounded history would eventually overflow the model’s context window. MessageWindowChatMemory enforces a sliding window: it keeps only the last N messages or trims based on token count. When the limit is exceeded, the oldest messages are dropped.
  • Context injection – The retrieved messages are typically added as system messages or as part of the prompt’s message list, depending on the model and advisor settings.

These strategies ensure that the model always receives the most relevant recent context without manual housekeeping.

MemoryAdvisor Integration​

MemoryAdvisor is the linchpin of the memory system. It implements RequestResponseAdvisor and is added to the ChatClient builder:

ChatClient client = ChatClient.builder(chatModel)
    .defaultAdvisors(new MemoryAdvisor(chatMemory))
    .build();

Internally, MemoryAdvisor uses an AdvisorContext to pass the conversation ID and, optionally, the number of messages to retrieve. The core logic:

  • In adviseRequest(): Retrieves messages from ChatMemory (via chatMemory.get(conversationId, lastN)), converts them to Message objects, and prepends them to the Prompt’s message list.
  • In adviseResponse(): Extracts the last user message and the assistant response from the ChatResponse, then calls chatMemory.add(conversationId, userMessage, assistantMessage).

The advisor also integrates with MessageWindowChatMemory to trim the history before retrieval. All of this happens behind the scenes; the application simply sets the conversation ID via ChatClient’s request advisor parameters:

chatClient.prompt()
    .user("My question")
    .advisors(a -> a.param("chat_memory_conversation_id", "user-123"))
    .call()
    .chatResponse();

ChatClient Integration​

ChatClient itself has no memory‑specific code. Memory is injected solely through the advisor mechanism. This design:

  • Keeps the core API uncluttered.
  • Allows memory to be disabled by simply not adding the MemoryAdvisor.
  • Enables different memory implementations (in‑memory, JDBC, Cassandra) without code changes.

The ChatClient builder’s defaultAdvisors() method accepts the MemoryAdvisor. The ChatClientRequestSpec also allows per‑request advisor parameters, so the conversation ID can be set dynamically.

Persistence Architecture​

Spring AI provides three built‑in ChatMemory implementations:

  • InMemoryChatMemory – Uses a ConcurrentHashMap<String, List<Message>>. Fast, simple, and suitable for development. Messages are lost on restart.
  • JdbcChatMemory – Stores messages in a relational database. The schema is managed via a ChatMemoryRepository that uses Spring’s JdbcTemplate. It supports transactional writes and persistent storage.
  • CassandraChatMemory – For distributed, horizontally scalable deployments. It leverages the Cassandra driver to store messages keyed by conversation ID, with time‑based partitioning.

The repository abstraction (ChatMemoryRepository) is internal; JdbcChatMemory and CassandraChatMemory implement a common pattern but are separate classes, not a unified SPI. However, the design is straightforward enough that a custom implementation only needs to implement the ChatMemory interface.

Context Window Management​

The MessageWindowChatMemory decorator implements context window constraints. It wraps any ChatMemory and limits the history returned by get():

ChatMemory memory = new InMemoryChatMemory();
ChatMemory windowedMemory = new MessageWindowChatMemory(memory, 20); // last 20 messages

Internally, MessageWindowChatMemory examines the message list and applies one or both of the following policies:

  • Count‑based window – Retain only the last N messages.
  • Token‑based window – (Future or advanced configuration) Truncate based on estimated token count using a TokenCountEstimator.

This decorator is automatically employed by the MemoryAdvisor if a MessageWindowChatMemory bean is configured. When the advisor retrieves history, it can specify a lastN parameter; the window implementation enforces the limit.

Design Patterns Used​

  • Strategy Pattern – ChatMemory is the strategy; InMemoryChatMemory, JdbcChatMemory, and CassandraChatMemory are concrete strategies. The advisor uses the strategy polymorphically.
  • Repository Pattern – Persistent implementations encapsulate data access behind the ChatMemory interface, akin to a repository.
  • Decorator Pattern – MessageWindowChatMemory wraps another ChatMemory to add windowing behavior without modifying the underlying store.
  • Advisor Pattern – MemoryAdvisor integrates memory into the advisor chain, following the same contract as other cross‑cutting advisors (logging, RAG, etc.).
  • Dependency Injection – The memory implementation is injected into the advisor, which is injected into the ChatClient. This promotes loose coupling.

Extension Points​

Developers can customize the memory system at several levels:

  • Custom ChatMemory implementation – Implement the interface and provide it as a Spring bean. For example, a Redis‑backed memory.
  • Custom windowing policy – Extend MessageWindowChatMemory or create a new decorator that implements different truncation logic (e.g., token‑based or semantic summarization).
  • Custom advisor behavior – Extend MemoryAdvisor and override adviseRequest() or adviseResponse() to modify how history is injected or what is saved.
  • Conversation ID management – The application or a higher‑level component determines how conversation IDs are generated and managed; the framework is unopinionated.

All extensions are standard Spring beans; no framework internals need to be modified.

Enterprise Best Practices​

  • Session isolation – Use distinct conversation IDs per user or session. In a multi‑user system, the ID must be scoped appropriately (e.g., "user:" + userId).
  • Multi‑user support – The memory store should partition data by conversation ID; persistent implementations inherently support this.
  • Scalability – For high‑throughput systems, use CassandraChatMemory or a custom Redis implementation. Avoid the in‑memory store in production.
  • Persistence – Choose a durable backend to survive application restarts. JDBC and Cassandra provide durability.
  • Security – Do not store sensitive personal data in plain text in the memory store. The advisor can be extended to mask or encrypt data before persisting.
  • Sensitive data handling – Be aware that conversation history may contain PII; implement data retention policies and automatic purging.
  • Performance optimization – Limit the window size to the minimum necessary. Large histories increase token consumption and storage overhead.

Performance Considerations​

  • Memory growth – The history for each conversation can grow unboundedly; windowing policies are essential to control storage and retrieval costs.
  • Token consumption – Every stored message adds to the prompt length. A sliding window of 20 messages can consume thousands of tokens; choose a size appropriate for the model’s context window.
  • Storage overhead – Persistent backends (JDBC, Cassandra) add I/O latency for each retrieval and write. Batch writes or asynchronous persistence can mitigate this, but the current advisor writes synchronously.
  • Repository performance – InMemoryChatMemory is O(1) for both reads and writes. JdbcChatMemory uses indexed queries; performance depends on database indexing.
  • Thread safety – InMemoryChatMemory uses a ConcurrentHashMap, so it is safe for concurrent access. Persistent implementations rely on the underlying database’s concurrency control.
  • Horizontal scalability – With a shared persistent store (Cassandra, JDBC with a shared database), multiple application instances can share the same conversation history.

Source Code Reading Guide​

To understand the memory subsystem, follow this order:

  1. ChatMemory.java – The core interface. Simple and clear.
  2. InMemoryChatMemory.java – The simplest implementation; study it to see the data structure (ConcurrentHashMap).
  3. MessageWindowChatMemory.java – The decorator that adds sliding window logic.
  4. MemoryAdvisor.java – Where the integration with the advisor chain happens. Trace how it retrieves and stores messages.
  5. JdbcChatMemory.java – If interested in persistent storage, see how JDBC is used.
  6. CassandraChatMemory.java – For distributed storage.

The unit tests (e.g., MemoryAdvisorTests) illustrate the lifecycle and are excellent for debugging.

Summary​

Spring AI Memory elegantly solves the statelessness of LLMs by injecting conversation history through the advisor chain. The separation of ChatMemory storage from MemoryAdvisor orchestration, combined with windowing decorators, gives developers a flexible, enterprise‑ready memory architecture. Understanding this source code deepens your ability to build long‑running, context‑aware AI applications while maintaining the portability and extensibility that Spring AI promises.

Continue to Advisor Source Code Analysis to see how the advisor chain itself is implemented.