Agent Memory Architecture: Working, Episodic, Semantic, and Procedural Memory for Multi-Agent Orchestration

Executive Summary

An agent that forgets everything between turns cannot hold a real job. Memory is what turns a stateless model call into something that behaves like it has continuity — remembering a user's preferences, recalling how a similar request was handled last month, and knowing the steps of a workflow without being re-taught them every time. This is a genuinely different problem from RAG: RAG retrieves from a relatively static external corpus, while memory captures state the agent itself generates and needs to carry forward.

This article breaks down the four memory types production agent systems use — working, episodic, semantic, and procedural — covers how multi-agent systems scope shared versus private memory to avoid context bloat and cross-contamination, addresses the decay and security concerns that come with persistent memory, and walks through what this looks like inside a multi-agent expense approval workflow.

Introduction

In enterprise environments, "the agent forgot what we told it" is one of the most common complaints in early Agentic AI deployments, and it is almost never a model capability problem — it is a missing memory architecture. Even models with very large context windows benefit from structured memory rather than relying on stuffing an ever-growing conversation into the prompt, because unmanaged context degrades response quality, increases latency and cost, and eventually exceeds any window regardless of size.

Production memory design borrows from research on cognitive architecture, most notably the CoALA framework, which separates an agent's memory into distinct types rather than treating it as one undifferentiated store.

The Four Memory Types

Working Memory

Working memory is the current session's active context — the conversation so far, recent tool results, and whatever state is needed for the immediate task. Even with a large context window, production systems actively manage working memory rather than letting it grow unbounded: clearing stale tool results once they've been used, maintaining a running summary or a simple running notes file the agent updates as it works, and compacting older parts of a long session into a condensed form rather than carrying the full transcript forward indefinitely.

Episodic Memory

Episodic memory captures specific past events with their context intact — what happened, when, and how it was resolved. For an agent that has handled a similar request before, episodic memory is what allows it to recall that a comparable case took a particular path last time, rather than solving every request as if it were the first of its kind. This is distinct from semantic memory: episodic memory is about specific events, not general facts.

Semantic Memory

Semantic memory stores general facts and preferences that hold true independent of any specific event — a user's stated preference for email over SMS notifications, an organisation's approval threshold for a given expense category, a customer's product tier. This is the memory type most similar in spirit to a user profile, and it is what lets an agent greet a returning user appropriately and apply known preferences without re-asking for them.

Procedural Memory

Procedural memory stores how to do something — the steps of a workflow, not facts about its subject matter. An agent handling expense approvals doesn't just need to know the approval threshold (semantic) or that a similar request was escalated last quarter (episodic); it needs to know the actual sequence of steps — validate the receipt, check the threshold, route to the right approver, log the decision — which is procedural knowledge that should persist independent of any single request.

Storage, Scoping, and Forgetting

Different memory types suit different storage technologies. Episodic and procedural memory, which are inherently relational (this event led to that decision, this step follows that one), fit naturally into graph databases that support fast relationship traversal. Semantic facts fit well into a conventional relational store where auditability and consistency guarantees matter. Working memory typically lives in-process or in a fast key-value store scoped to the active session.

Forgetting is a feature, not a gap. Retaining every interaction indefinitely degrades retrieval quality over time and creates unnecessary data retention exposure. Production memory systems apply:

• Temporal decay — older, unused memories lose retrieval priority
• Relevance scoring — memories that are rarely useful are deprioritised or archived
• Explicit retention policies — for example, discarding session-specific memory after a defined period, or clearing certain memory types when a workflow instance closes

In multi-agent systems specifically, memory needs both a shared and a private layer. Shared organisational or workflow memory holds state every agent in a workflow needs — the current status of a specific request, decisions already made. Private, agent-scoped memory (keyed by an agent identifier, separate from user or workflow identifiers) holds what one agent needs to do its own job without leaking into another agent's context. Skipping this separation is one of the more common causes of context bloat and cross-agent contamination in production multi-agent deployments, where one agent's working notes end up polluting another agent's reasoning.

The Current Tooling Landscape

As of mid-2026, the tooling around agent memory has matured quickly and is still moving.

• Mem0 has established itself as a widely adopted long-term memory layer with integrations across the major agent stacks.
• LangGraph provides native state persistence and resumable checkpoints for stateful multi-agent workflows.
• Google's Memory Bank, launched as part of the Gemini Enterprise Agent Platform, provides identity-scoped persistence so an agent can carry a user's preferences and history across sessions.

Given how quickly this space is moving, treat specific product capabilities as a snapshot rather than a permanent feature set, and check current documentation before committing to a specific vendor's memory layer for a long-lived production system.

Security and Reliability Considerations

Persistent memory introduces a version of the same trust problem covered in Part 3 for retrieved documents: a memory write is content the agent is choosing to trust and carry forward, and a bad write can compound. If an agent writes an incorrect fact to semantic memory — a wrong approval threshold, a misremembered preference — that error persists and silently influences every future interaction until something catches it, rather than being a one-time mistake.

Memory writes deserve validation before they're committed, not just before they're read: a memory manager layer that checks new writes for consistency with existing high-confidence memories, flags contradictions rather than silently overwriting, and scopes write permissions so one agent cannot arbitrarily rewrite another agent's private memory.

Privacy scoping matters just as much in multi-tenant or multi-user systems: semantic and episodic memory tied to one user must never leak into another user's session, which means memory keys need to include user or account identifiers as a first-class part of the schema, not an afterthought bolted on later.

A Multi-Agent Workflow, Start to Finish: Expense Approval

Marcus submits a travel expense report through the internal system on a Friday afternoon. A planner agent picks up the request and checks procedural memory for the expense approval workflow: validate receipts, check against policy thresholds, route to the appropriate approver, log the outcome. It doesn't need to be told these steps — they persist as procedural memory shared across every run of this workflow, not re-derived from scratch each time.

The planner hands off to a retrieval agent, which checks semantic memory and finds that Marcus's department has a standing threshold before a manager's sign-off is required, and that Marcus himself is flagged as a frequent international traveller — a preference-like fact stored from past interactions, not something re-derived from the current request. The retrieval agent also checks episodic memory and surfaces something useful: a similar multi-currency expense report from Marcus two months ago was initially flagged incorrectly due to a currency conversion error, and the correction that was applied then is available to prevent the same mistake from repeating.

An approval agent picks up the assembled context — written to shared workflow memory so every agent in this run can see the current state without re-fetching it — and routes the request to Marcus's manager, since it exceeds the department threshold. Once approved, a notification agent, working from its own private memory of Marcus's stated preference for email over in-app alerts, sends the confirmation the way he actually wants to receive it, not the system default.

Before any of this gets written back as a new episodic memory — "expense report from Marcus, multi-currency, approved after manager review" — the memory manager checks it against existing records for consistency. If a contradictory memory already existed (for instance, a prior note that Marcus's department threshold had just changed), the write is flagged for review rather than silently accepted, exactly the same discipline that governs a suspicious document in a RAG pipeline.

Two weeks later, the specific session-level working memory from this exchange is allowed to decay, while the useful episodic and semantic facts persist — the currency-handling correction is worth keeping, the specific Friday-afternoon timestamp is not.

Where Memory Ends and RAG Begins

It's worth restating the distinction from earlier in this series because it is easy to blur in a system that uses both. RAG retrieves from a relatively static external corpus — policy documents, product catalogs, knowledge base articles — that the agent did not generate itself. Memory captures state generated during the agent's own operation — what happened in past interactions, what preferences were learned, how a workflow is structured.

A production system typically uses both together: RAG to ground the approval agent's understanding of company expense policy, and memory to track Marcus's specific history and the workflow's own procedural steps. Neither replaces the other, and conflating them is a common design mistake that leads teams to try to solve a continuity problem with a retrieval fix, or vice versa.

Best Practices

• Treat the four memory types as genuinely different stores with different access patterns, not one generic key-value blob.
• Scope memory keys by user, agent, and workflow identity from the start — retrofitting this scoping later is expensive and error-prone.
• Validate memory writes for consistency before committing them, not just memory reads before using them.
• Apply explicit decay and retention policies; unmanaged memory growth degrades retrieval quality and increases data exposure.
• Give shared workflow memory and private agent memory clearly separate namespaces in any multi-agent system.

Anti-Patterns

• Relying on a large context window as a substitute for structured memory, letting session state grow unbounded.
• Sharing one memory space across all agents in a multi-agent workflow, causing cross-contamination between unrelated reasoning threads.
• Writing new memories without any validation step, allowing a single bad inference to silently corrupt future behaviour.
• Treating memory and RAG as interchangeable, leading to a continuity problem being patched with a retrieval fix that doesn't actually address it.

Frequently Asked Questions

Isn't a large context window enough to avoid needing structured memory?

No. Even with a very large window, unmanaged context degrades response quality and increases cost and latency well before the window's technical limit is reached, and it does not persist across sessions on its own. Structured memory with deliberate compaction and decay outperforms simply letting context grow.

How is episodic memory different from a RAG-indexed document?

A RAG-indexed document is external content the agent did not generate. Episodic memory is a record of the agent's own past interactions and outcomes. They can be stored in similar underlying technology, but they answer different questions — one is "what does this document say," the other is "what happened last time."

Do all four memory types need to be implemented for every agent?

No. A single-turn, single-session agent may need only working memory. Multi-session, multi-agent, or workflow-driven agents are where episodic, semantic, and procedural memory earn their cost — implement each type based on whether the agent actually needs continuity of that specific kind.

Key Takeaways

• Working, episodic, semantic, and procedural memory solve different continuity problems and should be treated as distinct stores.
• Multi-agent systems need both shared workflow memory and private per-agent memory, scoped by identity, to avoid contamination.
• Forgetting is a deliberate design feature — decay and retention policy prevent both quality degradation and unnecessary data exposure.
• Memory writes need the same validation discipline as RAG document ingestion — a bad write compounds silently over time.
• Memory and RAG solve different problems and are typically used together, not as substitutes for each other.

References for Further Reading

• Sumers et al., "Cognitive Architectures for Language Agents" (CoALA) — the framework underlying the working/episodic/semantic/procedural memory taxonomy.
• Anthropic, "Effective Context Engineering for AI Agents" — covers compaction and working memory management in depth.
• Mem0, LangGraph, and Google's Memory Bank documentation — check current product documentation directly, as this tooling landscape is evolving quickly.

Coming Next

Part 5 covers fine-tuning with LoRA and QLoRA — the first method in this series that actually changes model weights — using a high-volume structured data extraction agent as the working example.