AI Agent Memory

Short-term memory

What is a short-term memory?

Short-term memory refers to the temporary context a large language model (LLM) can access during a single session. It includes the active conversation such as the user prompts and model responses. Short-term memory is not persistent; it is limited by the model’s context window size, which may be dependent on the model. When the context window is met, the older parts of the exchange are printed or truncated – especially forgetting them. Short-term memory is designed to reset when the interaction ends, unless it is instructed to re-include prior context.

Why is short-term memory for context retention important in large language models and applications?

Short-term memory is critical for session-aware behavior. In a single session, models can maintain coherence, follow instructions, and build on prior context. Unlike long-term memory, short-term conversation history exists only during an active session.

Long-term memory

What is a long-term memory?

Long-term memory refers to a system’s ability to store and retrieve information across sessions. It is typically implemented at the application level and functions similarly to an external database or knowledge store that can be queried or updated.

This memory category often requires user authentication to ensure it is scoped out correctly. If it is not configured for user authentication, then it is tied to the global memory, which can cause cross-user data leaks.

Why is long-term memory for context retention important in large language models and applications?

Long-term memory enables persistent context retention so that the large language model (LLM) or application can recall facts, preferences, or prior interactions over time. Unlike short-term memory, long-term memory conversation history can exist across multiple sessions. It is essential for personalization across sessions, allowing the model to offer more relevant, consistent, and tailored responses over time.

‍