Build · Topic 04 of 14
AI agent memory and context: what an agent remembers, and how to manage it
Agent memory is what an AI agent remembers between turns and across sessions, a different thing from the RAM it runs on. Short-term memory is the context window: the instructions, conversation and tool results the model can see. Long-term memory is a separate store: facts about a user, past tasks and learned procedures. You own its retention, deletion and isolation.
In this guide: 83 pages
Key takeaways
- Short-term memory is the context window and costs tokens on every call. Long-term memory is a store with a pipeline: extract, consolidate, store, retrieve. The two need different controls.
- On 2 Nov 2026 Amazon Bedrock AgentCore Memory and Google’s Agent Platform Memory Bank were generally available; memory in Microsoft Foundry Agent Service and Microsoft 365 Copilot memory were still preview.
- Microsoft 365 Copilot memory sits outside Purview retention and audit, and deleting a chat does not delete the memories made from it. Plan the erasure path before you switch it on.
- Memory can be poisoned through ordinary conversation (MINJA reports a 98.2% injection success rate with query-only access; OWASP lists it as ASI06). Screen what enters memory and scope what leaves it.
- Start without long-term memory. Add session memory, then user preferences with a TTL, then episodic or procedural memory only with per-user scoping and a review gate.
Why does agent memory matter, and why is it a risk?
Memory matters because an agent without it starts every task from zero and cannot carry a job across sessions. It is a risk because a store of what people said is personal data, it can be poisoned through conversation, and in most products it is harder to delete than the conversation that produced it.
The product trend runs one way. Microsoft, AWS and Google each ship a managed long-term memory service for agents, Anthropic and OpenAI ship memory and context-management features in their APIs, and ChatGPT has referenced past chats since 10 Apr 2025. Anthropic reports a 39% improvement on its internal agentic-search evaluation with the memory tool and context editing together (vendor-reported, 29 Sep 2025).
The risk side is less advertised. A memory store holds what employees and customers told the agent, so GDPR applies in full. Microsoft’s page for Microsoft 365 Copilot memory (2 Sep 2026) says Purview retention policies and labels do not apply to it, memory actions generate no audit log entries, admins cannot restrict what gets remembered, and deleting a chat does not delete the saved memories made from it.
Then there is poisoning. The MINJA paper (arXiv 2503.03704, revised 12 Feb 2026) shows an attacker can plant false records through ordinary queries, with no write access. OWASP’s Top 10 for Agentic Applications (9 Dec 2025) lists this as ASI06, Memory and Context Poisoning.
How does agent memory work?
Memory works in two tiers. Short-term memory is the context window, saved between turns and replayed. Long-term memory is a pipeline that extracts facts from the conversation, consolidates them, stores them with embeddings and metadata, and retrieves them by similarity on later turns.
- Working memory is the context window. Everything the model reasons over has to fit in the prompt. Windows have grown (Anthropic made a 1M-token window generally available on 13 Mar 2026), but models use the middle of a long input worse than the ends, as the Lost in the Middle paper measured in 2023.
- Session state is saved and replayed: a LangGraph checkpointer, an OpenAI Conversations object, or AgentCore short-term memory, a stream of immutable, timestamped events grouped by actorId and sessionId, kept up to 365 days with the expiry fixed per event at write time.
- The window is kept from overflowing. Anthropic’s context editing clears old tool results on the server (by default at 100,000 input tokens, keeping the last three tool uses); compaction replaces older turns with a model-written summary, from Anthropic (beta, 14 Sep 2026) and OpenAI.
- Long-term memory follows one pipeline everywhere: extract, consolidate, store, retrieve. A model pulls facts, preferences or summaries out of the events; consolidation merges duplicates and resolves conflicts; records are stored with embeddings and metadata; later turns retrieve them by similarity. AWS adds a reflection step for episodic memory.
- Writes mostly happen in the background. AgentCore says extraction can take a minute or more, and episodic records appear only once an episode has finished, so a fact told to the agent may not be recallable in the next session.
- Isolation is a key you assign. AgentCore scopes by actorId, sessionId and namespace paths such as /users/{actorId}/preferences/ (the managed harness assigns the actor id for you); Foundry requires a scope on every low-level API call and resolves it from the caller’s identity only when the memory search tool uses {{$userId}}. A wrong scope is a data leak.
What types of agent memory are there?
Four types, from a taxonomy the CoALA paper set out in 2023 and the vendors now use: working memory (the context window), semantic memory (facts about the user and the world), episodic memory (what happened in past tasks) and procedural memory (how to do things). Each vendor names them differently.
The four memory types, what each holds, an enterprise example, and the product names on 2 Nov 2026.
| Type | What it holds | Enterprise example | Product names |
|---|---|---|---|
| Working | The current thread, instructions, tool results | A procurement agent holds the purchase order it is checking and the supplier record it just fetched | Context window; LangGraph thread state; AgentCore short-term events; Agent Platform Sessions |
| Semantic | Facts and preferences about a user or the world | A service-desk agent remembers that an engineer works the night shift and prefers tickets in German | AgentCore semantic and user preference strategies; Foundry user profile memory; Copilot saved memories; Memory Bank memory profiles |
| Episodic | What happened in past tasks, often reused as examples | An IT agent recalls how it resolved the same VPN error last month and what the user confirmed | AgentCore episodic strategy with reflections (Dec 2025); AgentCore summary strategy; Foundry chat summary memory |
| Procedural | How to act: routines, rules, refined instructions | An accounts-payable agent learns that one supplier’s invoices need a purchase-order match before approval | Foundry procedural memory (preview, 2 Jun 2026); the system prompt; Anthropic memory tool files; LangMem prompt optimisation |
The types of agent memory sub-page goes through each type with its controls; in short, semantic memory is the first long-term type most agents need, and procedural memory most needs a review gate.
Agent memory vs RAG vs context engineering: what is the difference?
Memory is what the agent learned from its own interactions; retrieval-augmented generation (RAG) is how it looks things up in company content it did not write; context engineering is the discipline of deciding what from either goes into the window on a given call.
Three neighbouring terms.
| Term | Source of the content | Who writes it | Typical store | Controls that matter |
|---|---|---|---|---|
| Agent memory | The agent’s own past interactions | The agent, through an extraction model | Memory store (AgentCore Memory, Foundry memory, Memory Bank) | Scope per user, TTL, erasure, poisoning screen |
| RAG and knowledge | Documents, wikis, tickets, databases | Your organisation | Search index or knowledge base (Foundry IQ, Bedrock Knowledge Bases) | Permission-aware retrieval, ACL sync, citations |
| Context engineering | Both, plus instructions and tool results | The developer, per call | The context window itself | Token budget, ordering, compaction, tool loading |
Microsoft’s own guidance draws the line the same way: memory for user-specific context learned over time, a Foundry IQ knowledge base for curated organisational content.
Which memory products exist beyond the three clouds?
Five families matter on 2 Nov 2026: LangGraph and LangMem for open-source state and memory, Mem0 and Letta as memory layers, and the memory features of the Anthropic and OpenAI APIs. Our research found no shared record format among them, so moving memory between products is a custom export.
Memory products outside the three cloud services, as seen 2 Nov 2026.
| Product | What it gives you | Status and notes |
|---|---|---|
| LangGraph checkpointers and stores, LangMem | Thread state through a checkpointer; a namespaced JSON store for long-term memory; LangMem adds extraction, consolidation and prompt optimisation, on the hot path or in the background | Open source (LangGraph 1.0 on 22 Oct 2025); versions not checked |
| Mem0 | A memory layer with open-source and hosted editions; integrations with Strands Agents (announced with AWS on 19 May 2025), ElastiCache for Valkey and Neptune Analytics | Live; the company’s compliance claims are self-reported |
| Letta (formerly MemGPT) | Editable memory blocks inside the context, shared across agents; sleep-time agents reorganise memory in the background (April 2025) | Live; Bilt runs per-user recommendation agents on it (Letta case study, undated) |
| Anthropic memory tool, context editing, compaction, Managed Agents memory | File-based memory under a /memories path that your application executes against storage you control; server-side clearing of old tool results; summarisation of older turns; managed memory stores for Claude Managed Agents | Memory tool stable and available on Bedrock, Google Cloud and Microsoft Foundry; context editing, compaction and Managed Agents memory in beta |
| OpenAI Conversations API, Agents SDK Sessions, server-side compaction | Durable conversation objects; session history in SQLite, Redis, SQLAlchemy, MongoDB or Dapr back ends; compaction at a token threshold | Live; the Assistants API that preceded it shut down on 26 Aug 2026 |
Two retirements show why the record format matters: Amazon Bedrock Agents Classic closed to new customers on 30 Jul 2026 with AWS pointing at AgentCore Memory, and the OpenAI Assistants API’s threads gave way to Conversations on 26 Aug 2026. Budget for an export on every platform change.
What are the enterprise concerns: retention, PII and the right to erasure?
Three questions decide whether a memory store is lawful to run: how long each copy of a memory lives, what personal data the extraction model may write, and whether you can find and delete one person’s records across every copy. The products answer them differently.
Retention. AgentCore short-term events expire after up to 365 days (the CLI accepts 3 to 365, default 30), with the expiry fixed per event at write time, so shortening it later leaves old events untouched; long-term records never expire unless you set a TTL per strategy, minimum 31 days. Foundry’s latest preview adds a store-level default TTL and item-level delete. In Microsoft 365, Purview retention policies have separate locations for Copilot experiences and enterprise AI apps, but Copilot memory itself is outside them: retention policies and labels do not apply, and there are no admin controls to enforce retention on memory (Microsoft Learn, 2 Sep 2026).
PII. The extraction model decides what to keep, and admins cannot restrict what Microsoft 365 Copilot adds to memory. The screening tools sit at the boundary: Azure AI Content Safety prompt-injection detection on what enters and leaves Foundry memory, Amazon Bedrock Guardrails sensitive-information filters that block or mask PII, and Google’s Model Armor with Sensitive Data Protection templates. Note the gap AWS documents: Guardrails PII masking does not cover model invocation logs or tool-call arguments and results, which is where PII moves.
Erasure. GDPR Article 17 needs item-level find, export and delete across raw events, extracted records, summaries and reflections. On Microsoft 365, admins search memories through eDiscovery as item class IPM.Contact in the CopilotMemory folder and purge them through Graph Explorer, 10 items per mailbox per run, holds removed first; deleting a conversation leaves the associated memory in place, and custom instructions are not discoverable at all. On AWS, scope namespaces per user so a person’s records can be located; per-record delete APIs were not verified in our research. Client-side memory such as Anthropic’s memory tool puts deletion entirely on you.
Two more duties. Run a DPIA that names memory and logs as processing activities, and do not call embeddings or summaries anonymous without meeting the EDPB’s criteria of no singling out, no linkage and no inference (draft Guidelines 02/2026, 8 Jul 2026).
What is memory poisoning, and how do you defend against it?
Memory poisoning is an attack that plants false facts or hidden instructions in an agent’s persistent memory so that later sessions, often for other users, act on them. Unlike prompt injection, the payload survives the conversation that delivered it.
The research is specific. MINJA (Memory Injection Attacks on LLM Agents via Query-Only Interaction, arXiv 2503.03704) injects records through ordinary queries alone and reports a 98.2% average injection success rate and a 76.8% average attack success rate across three agents. OWASP lists ASI06 Memory and Context Poisoning; MITRE ATLAS tracks AI Agent Context Poisoning as AML.T0080 (version 5.0.0, 30 Sep 2025). Microsoft’s Foundry memory page names the same threat and tells you to validate everything entering or leaving the memory system and to run adversarial tests.
The defences we install, in order. Screen writes: the same prompt-injection and PII checks on candidate memories as on user input. Scope reads: one namespace per user, and reflections kept at the actor level, because AWS warns that episodic reflections can span multiple actors. Keep tenant keys out of the model’s hands: AgentCore’s strictly consistent metadata (15 Jun 2026) copies tenant keys unchanged from events to records and never merges records with different values. Expire: a TTL on every long-term strategy. Review procedural changes as prompt changes. Test: poisoned-memory cases in red-team runs, and an alert on memory writes that arrive with instructions in them.
What we refuse to do: let an agent write to memory shared across users without a human review step, or treat a memory store as trusted input. Retrieved memories are context, and the system prompt should say so.
How do Microsoft, AWS and Google implement it?
The same capability on each cloud, with names and statuses as the vendor pages showed on 2 Nov 2026.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Managed memory service | Memory in Foundry Agent Service and the Memory Store API: public preview since 25 Nov 2025, refreshed at Build (2 Jun 2026) | Amazon Bedrock AgentCore Memory: generally available since 13 Oct 2025 | Agent Platform Memory Bank and Agent Platform Sessions (renamed from Vertex AI Agent Engine Memory Bank and Sessions on 22 Apr 2026): generally available since December 2025 |
| Short-term or session memory | Threads managed by the agent framework; chat summary memory for the distilled form | Immutable events by actorId and sessionId, kept up to 365 days, expiry fixed at write time | Agent Platform Sessions; IngestEvents streaming (GA 8 Jul 2026) |
| Long-term memory types | User profile, chat summary, procedural (procedural added 2 Jun 2026, preview) | Semantic, user preference, summary, episodic with reflections; built-in with overrides; self-managed | Gemini-based extraction and consolidation; memory profiles (GA 15 Jul 2026); revision history |
| Isolation | A scope per user; {{$userId}} with the memory search tool; 100 scopes per store, 10,000 memories per scope | Namespaces per actor and session; strictly consistent metadata for tenant keys (15 Jun 2026); resource-based policies (March 2026) | IAM Conditions, VPC Service Controls, customer-managed encryption keys |
| Retention and deletion | Store-level default TTL and item-level delete (preview); Copilot memory outside Purview retention, found through eDiscovery as IPM.Contact | Event expiry fixed at write time; record TTL of 31 days or more per strategy, otherwise no expiry; customer-managed KMS keys | Revision history; per-memory deletion not checked in this review |
| End-user assistant memory | Microsoft 365 Copilot memory (saved memories, chat-history details, custom instructions): preview on the Learn page of 2 Sep 2026; stored in the user’s Exchange mailbox; Enhanced personalization control, on by default | None in Bedrock itself; memory belongs to each application | Gemini Enterprise memory controls not reviewed in this pass |
Microsoft’s Message Center post MC1158329 (revised 1 Sep 2026) points to general availability of Copilot chat-history personalisation from mid-September 2026, while the Learn page of 2 Sep 2026 still says preview, so this page calls it preview.
When should you not give an agent memory, and what goes wrong?
Do not add long-term memory to an agent that completes its task in one session, to an agent that serves many people through one identity, or to anything that handles special-category data until the erasure path has been tested. Once memory is on, these are the failures we see.
Leakage between users or tenants
Shared or cross-actor memory carries one user’s facts into another’s answers. AWS warns that episodic reflections can span multiple actors; in Foundry a wrong scope on a low-level call is an isolation bug.
Memory that outlives the chat
Users assume deleting a conversation deletes what the agent learned. In Microsoft 365 Copilot it does not, and AgentCore long-term records never expire without a TTL. Tell users what persists.
Stale and conflicting memories
Preferences change, and an inference can be wrong from the start. Consolidation, TTLs and revision history each help; none replaces a way for the user to see and correct what is stored.
Compaction that drops the one fact that mattered
A summary of older turns can lose a constraint a later turn needs; Anthropic documents pairing compaction with a memory store for facts that must survive.
Latency and freshness
Background extraction takes time. AgentCore says a minute or more, and episodic records wait for the episode to end, so a user who opens a new session a minute later may find the fact forgotten.
Preview dependencies and no portable format
Foundry memory is preview with no VNet support and an API that can change; Copilot memory is preview. Our research found no cross-vendor memory record format, so a platform move is a custom export.
What should you set up?
The controls that make a memory store safe to run, in the order we install them.
- A written statement of what the agent remembers, for whom, how long and where, shown to users.
- Session memory first: thread state that expires with the task, before any long-term type.
- One scope per user: actorId namespaces on AWS, scope with {{$userId}} on Foundry, user IDs in LangGraph store namespaces. No shared namespace without a review step.
- A TTL on every long-term strategy at creation (31 days or more on AgentCore; store default TTL on Foundry), because records written without one never expire.
- Screening on the write path: prompt-injection detection and PII filters applied to candidate memories, with the same policy as user input.
- An erasure runbook tested with a real request across events, records, summaries and reflections; on Microsoft 365, the eDiscovery search for IPM.Contact plus a Graph Explorer purge, holds removed first.
- Poisoned-memory cases in the evaluation set and the red-team plan, and an alert on memory writes that contain instructions.
- A registry entry for the memory store: owner, data classes, retention, review date, DPIA reference.
Enterprise example · Worked example · illustrative
A 9,000-person logistics company gives its customer-service agent memory
Illustrative scenario. A European freight and logistics provider with 9,000 staff, a customer-service agent on Amazon Bedrock AgentCore that handles shipment queries for 400 business customers, and a GDPR duty to delete a contact’s data on request.
The first version had no long-term memory, so customers repeated their account number, preferred depot and language on every call. The product owner asked for memory; the data protection officer asked what would be stored, for whom, and how it would be deleted. Both questions were answered first.
The team enabled AgentCore Memory with short-term events kept 30 days and one long-term strategy. The user-preference strategy wrote to /users/{actorId}/preferences/ with a 90-day record TTL. Episodic memory was left off: reflections can span actors, and the business case did not need it. Each business customer was a tenant key carried as strictly consistent metadata, so the model could never merge two customers’ records.
Every candidate memory passed the same Bedrock Guardrails sensitive-information filter as user input before it was stored, and a second check held back candidate memories that contained instructions for human review. The system prompt told the agent to treat retrieved memories as background and never as instructions.
Erasure was tested before launch with a dummy contact: the runbook located the actor’s events by actorId and the preference records by namespace, the deletion calls were tested against the AWS APIs, and the trace showed no copy elsewhere. The memory store got its own registry entry with an owner, data classes, the 90-day TTL and the DPIA reference. Six months in, the DPO had closed two erasure requests from the runbook without an engineer.
- Owner
- Customer-service product owner; the data protection officer signs the memory statement and the DPIA
- Identity
- AgentCore Identity workload identity for the agent; actorId from the user the portal authenticated
- Risk tier
- Tier 2: reads customer and shipment data, stores preferences; no episodic or procedural memory
- Approval
- Memory configuration reviewed as a prompt change; candidate memories with instructions routed to human review
- Data access
- Scope per actor; tenant key as strictly consistent metadata; Guardrails PII filter on the write path; customer-managed KMS key
- Retention
- Events 30 days; preference records with a 90-day TTL set at creation
- Monitoring
- CloudWatch generative AI observability traces of memory operations; alert on memory writes that contain instructions
- Retirement
- Memory store deleted with the agent; erasure runbook run for every customer that leaves; registry entry closed
How do you start?
- Write the memory statement first: what is remembered, for whom, how long, where, and how it is deleted. If you cannot write it, leave memory off.
- Ship session memory only and measure the repeat-question rate for a month.
- Add one long-term type, usually user preferences, with a scope per user and a TTL set at creation.
- Put screening on the write path and poisoned-memory cases in the evaluation set before the first production write.
- Test erasure with a dummy user and time it.
- Register the store, then review after Ignite and re:Invent in late 2026, when the Foundry and Copilot memory statuses are likely to change.
Frequently asked questions
Is AI agent memory the same as computer memory or RAM?
No. Agent memory means what an agent remembers: the context window it reasons over now and the store it keeps across sessions. RAM is the hardware memory the model runs on.
What is the difference between agent memory and RAG?
Memory is what the agent learned from its own interactions with a person; RAG (retrieval-augmented generation) is how it looks up company documents it did not write. They use similar search technology and need different controls: scope and erasure for memory, permission-aware retrieval for RAG.
Does deleting a Microsoft 365 Copilot chat delete its memory?
No. Microsoft’s page of 2 Sep 2026 says deleting a chat does not delete the saved memories generated from it, Purview retention policies do not apply to memory, and memory actions create no audit entries. Admins find and delete memories through eDiscovery (item class IPM.Contact) and Graph Explorer.
Is memory in Microsoft Foundry generally available?
No. On 2 Nov 2026 memory in Foundry Agent Service and the Memory Store API were in public preview (first preview 25 Nov 2025, procedural memory added 2 Jun 2026), with no VNet support.
What is memory poisoning in AI agents?
An attack that plants false facts or hidden instructions in persistent memory so later sessions act on them. Research (MINJA, 2025 to 2026) shows it works through ordinary queries with no write access. OWASP lists it as ASI06 in its agentic top 10 for 2026. Screen writes, scope reads, set TTLs and test for it.
How do we handle a GDPR erasure request for agent memory?
Locate every copy: raw events, extracted records, summaries and reflections. On AWS, namespaces per user make records findable; on Foundry the preview has item-level delete; on Microsoft 365 use eDiscovery for IPM.Contact items plus a Graph Explorer purge, holds removed first.
In this topic
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- What is memory in Foundry Agent Service
- Introducing memory in Foundry Agent Service
- What’s new in Microsoft Foundry at Build 2026
- Manage Copilot personalization and memory
- Search for and delete AI application data in eDiscovery
- Learn about retention for Copilot and AI apps
- MC1158329: Microsoft 365 Copilot personalization and memory (Message Center mirror)
- AgentCore Memory strategies
- Create an AgentCore Memory
- AgentCore episodic memory strategy
- Memory on the AgentCore managed harness
- Amazon Bedrock AgentCore is now generally available
- AgentCore Memory adds metadata on long-term memory records
- AgentCore Memory strictly consistent metadata
- Amazon Bedrock Agents Classic maintenance mode
- Amazon Bedrock Guardrails sensitive information filters
- How Thomson Reuters built an agentic platform engineering hub with Amazon Bedrock AgentCore
- Memory Bank and Sessions generally available (Agent Builder blog; Vertex AI was renamed Gemini Enterprise Agent Platform on 22 Apr 2026)
- Gemini Enterprise Agent Platform release notes
- Memory tool
- Context editing
- Compaction
- Managing context on the Claude Developer Platform
- Claude Developer Platform release notes
- Conversation state
- Assistants API migration guide
- OpenAI updates ChatGPT to reference your other chats
- Memory (LangChain and LangGraph concepts)
- Sleep-time compute
- AWS and Mem0 partner to bring persistent memory to Strands Agents
- Cognitive Architectures for Language Agents (CoALA), arXiv 2309.02427
- Lost in the Middle: How Language Models Use Long Contexts, arXiv 2307.03172
- Memory Injection Attacks on LLM Agents via Query-Only Interaction (MINJA), arXiv 2503.03704
- OWASP Top 10 for Agentic Applications for 2026
- MITRE ATLAS data changelog
- Regulation (EU) 2016/679 (GDPR), Article 17
- EDPB sheds light on anonymisation and web scraping for generative AI
- EDPB guidelines, recommendations and best practices
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner