- Home
- AI agents
- Platforms and frameworks
- Reference architecture on Azure
Build · Platforms and frameworks
What does an enterprise AI agent reference architecture on Azure look like?
This reference architecture is a proposed design for running enterprise AI agents on Microsoft Azure and Microsoft 365. It places each part of an agent on a named service: Copilot Studio or Foundry for the agent, Entra Agent ID for identity, Foundry IQ for knowledge, API Management and Foundry tools for actions, and Agent 365 for governance.
In this guide: 83 pages
Key takeaways
- This is a proposed design assembled from Microsoft’s documented services; it describes no particular client system. Check each status before you build, because several parts are still preview.
- Choose the build surface per agent: Copilot Studio for makers and agent flows, Foundry Agent Service for pro-code prompt and hosted agents, and Agent Framework when you need your own orchestration.
- Each agent gets its own Entra agent identity, Conditional Access for agents applies policy to it, and retrieval runs under the user’s identity so permissions hold.
- Plan around the preview parts, such as Foundry guardrails on tool calls, Foundry memory and the Durable Extension. The last section lists a generally available fallback for each.
Which Azure service does each part of the agent run on?
Each part of the agent maps to one or two services. The table lists the service we would place there, its status on 2 Nov 2026 and the reason for the choice.
The proposed design layer by layer. Statuses as Microsoft’s pages showed them on 2 Nov 2026.
| Part | Service | Status | Why here |
|---|---|---|---|
| Low-code agent | Copilot Studio, with agent flows for fixed steps | Computer use GA May 2026; A2A GA Apr 2026 | Makers own the agent, and flows give deterministic steps |
| Pro-code agent | Foundry Agent Service prompt and hosted agents | Service GA May 2025; hosted agents GA per an Azure update of 2 Jun 2026, while Microsoft’s Build recap said early July | Your configuration or container, VNet options, publishing to Teams and Microsoft 365 Copilot |
| Orchestration | Agent Framework workflows; the Durable Extension for long waits | 1.0 GA 3 Apr 2026; durable hosting preview at 1.0 | Checkpoints, approvals and pause and resume in code |
| Tools | Foundry Toolbox; API Management for REST APIs as MCP tools; API Center as private MCP registry | Toolbox page carries no overall label; API Management MCP support GA 18 Nov 2025 | One MCP endpoint per agent, with policy on every call |
| Knowledge | Foundry IQ knowledge bases on Azure AI Search; Microsoft 365 Copilot Retrieval API | Knowledge bases GA 2 Jun 2026; native ACL and label enforcement in Search is preview | Retrieval under the caller’s Entra identity |
| Memory | Foundry memory, or a store of your own | Public preview, no VNet support | Only where a measured task needs it |
| Identity | Microsoft Entra Agent ID; Conditional Access for agents | Agent ID GA (Learn, 1 May 2026); Conditional Access for agents GA | One identity per agent, created in Foundry with zero permissions |
| Guardrails | Foundry guardrails; Azure AI Content Safety Prompt Shields; the API Management llm-content-safety policy | Agent guardrails preview; Prompt Shields GA since Aug 2024 | Checks on input, tool calls and output |
| Observability | Foundry tracing to Application Insights | Tracing GA for prompt and hosted agents; the Agents view in Application Insights is preview | One trace per run, with sensitive content in a protected table |
| Governance | Agent 365 registry; Purview audit, DLP and retention | Agent 365 GA 1 May 2026 | Inventory, owners and data controls in one place |
Channels sit above all of this. Agents from either surface publish to Teams and Microsoft 365 Copilot, and web or mobile apps call Foundry agents directly. Scheduled and event-triggered agents run with no user behind them, which makes them autonomous agents that need their own narrow grants. One rule holds across every layer: anything an agent can change, a person or a policy can see and stop.
How does one request move through the architecture?
From the channel to the agent, out through identity-checked retrieval and tools, and back with a trace. The numbered path follows an employee’s request to an agent built in Foundry and published to Teams.
- The employee asks in Teams. The agent runs in Foundry Agent Service under its own Entra agent identity, which Foundry provisioned with a federated credential, so no secret exists, and with no permissions until you grant them.
- Conditional Access for agents evaluates that identity, with templates such as blocking high-risk agent identities. When the agent acts for the employee it uses the on-behalf-of flow; delegated access for Foundry tools is still preview.
- The agent retrieves from a Foundry IQ knowledge base. The query runs under the caller’s Entra identity, so Azure AI Search returns only what the employee can open.
- The agent calls tools through a Foundry Toolbox or an API Management MCP endpoint. API Management validates the token and applies rate limits and the llm-content-safety policy, and Foundry guardrails can check the tool call and the tool response (both preview).
- A write waits for approval. Foundry’s MCP tool has a require_approval setting, and for long waits the Agent Framework Durable Extension suspends the workflow so the wait costs little.
- Each step lands in Application Insights as an OpenTelemetry trace. Since 30 Sep 2026, sensitive GenAI content in newly ingested data goes only to the AppGenAIContent table, first announced as a public preview in July 2026. Mark that table Protected so only the Privileged Monitoring Data Reader role can read it, review custom roles with the same rights, and note the temporary opt-out that runs to 30 Sep 2027.
- Agent 365 holds the registry record with owner and sponsor, and Purview audit records the interaction, including the agent ID, the model provider and any cross-prompt-injection flags.
Which decisions change the Azure design?
Four decisions reshape it: who builds the agent, whether it acts for a user or as itself, where data may be processed, and how isolated the network must be.
- Who builds. Makers in a business unit get Copilot Studio inside governed Power Platform environments; engineers get Foundry. Both appear in the Agent 365 registry, and they can call each other over A2A, which is generally available in both.
- Delegated or autonomous. A delegated agent never exceeds the user’s rights, and Purview data security policies for Foundry apply only when calls carry user context; app-only calls get audit without enforcement. An autonomous agent needs its own narrow grants and a tighter review.
- Data location. Microsoft 365 Copilot is an EU Data Boundary service for EU customers, but Microsoft states that models provided by Anthropic are currently excluded from the boundary, and Multi-Geo tenants fall outside it. Pin model choices per agent where residency matters.
- Network isolation. Foundry’s MCP tool reaches private servers through a VNet subnet, and hosted agents have network egress controls in preview. Foundry memory has no VNet support during its preview, so keep regulated data out of it.
What is still preview, and what do you build instead?
Several parts of the design were preview on 2 Nov 2026. For each one there is a generally available fallback to use until Microsoft releases it.
Preview parts and their fallbacks, as of 2 Nov 2026.
| Preview part | Use until it is generally available |
|---|---|
| Foundry guardrails on tool calls and tool responses | Prompt Shields on inputs and documents, plus API Management policies in front of tools |
| Foundry memory | Session state in your own store with an explicit TTL, or no long-term memory |
| Agent Framework Durable Extension | Agent Framework checkpoints, stable in 1.0, plus timers, scheduling and recovery from an orchestrator your team already runs |
| Native ACL and label enforcement in Azure AI Search | String filters on user and group IDs, built and tested in your own code |
| Delegated access for Foundry tools | Tools behind API Management that validate the user’s token, or Copilot Studio connectors that run with end-user credentials |
| Foundry multi-agent workflows | Agent Framework workflows in your own code, GA since 3 Apr 2026 |
| Application Insights Agents view and Foundry dashboards | Standard Application Insights queries over the same traces |
Microsoft Ignite in late November 2026 is likely to change some of these labels, and we review this page after it.
How do Microsoft, AWS and Google implement it?
The same layers on the other two clouds, for teams that run on more than one. The AWS design has its own page in this guide.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Agent runtime | Foundry Agent Service; Copilot Studio | AgentCore harness and AgentCore Runtime | Agent Runtime in Gemini Enterprise Agent Platform |
| Agent identity | Entra Agent ID | AgentCore Identity | Agent Identity (API preview since 18 Jun 2026) |
| Tool gateway and policy | Foundry Toolbox; API Management | AgentCore Gateway with AgentCore Policy | Agent Gateway with Model Armor |
| Registry | Agent 365 registry | AWS Agent Registry | Agent Registry |
Agent 365 registry sync with AWS and Google Cloud was in public preview on 2 Nov 2026, so a mixed estate still needs one registry declared as the system of record.
Frequently asked questions
Is this a Microsoft reference architecture?
No. It is our proposed design built from Microsoft’s documented services, with statuses as of 2 Nov 2026, and it describes no client deployment. Microsoft publishes guidance per service; use this page to see how the pieces fit, and check each service page before you build.
Do we need both Copilot Studio and Foundry?
Often yes, for different agents. Copilot Studio suits agents that makers build and maintain with agent flows; Foundry suits pro-code agents with custom tools, hosting and evaluation. Both register in Agent 365 and can call each other over A2A.
Where should agent memory live on Azure?
Only where a task needs it. Foundry memory is still public preview without VNet support, so for regulated data keep session state in your own store with an explicit TTL, and add long-term memory once the feature is generally available or the data allows it.
Which licences does the governance layer depend on?
Entra Agent ID is available to all Entra customers, but extending Entra security features to agents needs Agent 365, and Conditional Access for agents requires Microsoft 365 E7, or Agent 365 with Entra P1 or Microsoft 365 E3, according to Microsoft Learn. Check the licensing pages before you plan, since they changed several times in 2026.
How do Azure agents work with agents and tools on other clouds?
Through the open edges. Foundry agents call any A2A endpoint with the a2a tool, generally available on A2A 1.0, and reach remote MCP servers through the MCP tool. Agent 365 can sync registry records with AWS and Google Cloud in public preview, and Entra Agent ID has integration guides for agents on Amazon Bedrock. Keep one identity provider and one registry as the systems of record.
More in Platforms and frameworks
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- What is Microsoft Foundry Agent Service?
- Foundry Agent Service generally available (Azure update AZ494654)
- Foundry hosted agents now generally available (Azure update AZ563596)
- Harnesses in Copilot Studio
- What’s new in Copilot Studio
- Microsoft Agent Framework version 1.0
- Durable Extension (Microsoft Agent Framework)
- Foundry Toolbox overview
- Expose a REST API as an MCP server in Azure API Management
- Register and discover MCP servers in Azure API Center
- Model Context Protocol tool in Foundry Agent Service
- Agent-to-agent tool in Foundry Agent Service
- What is Foundry IQ?
- Document-level access control in Azure AI Search
- What is memory in Foundry Agent Service?
- What’s new in Microsoft Entra Agent ID
- Agent identity in Microsoft Foundry
- Conditional Access for agents in Microsoft Entra
- Guardrails and controls in Microsoft Foundry
- Prompt Shields in Azure AI Content Safety
- llm-content-safety policy
- Set up tracing for Foundry agents
- Restrict access to sensitive content in traces
- Agents view in Application Insights
- Microsoft Agent 365 overview
- Microsoft Agent 365 now generally available
- Audit logs for Copilot and AI applications
- Microsoft Purview for Microsoft Foundry
- Data, privacy and security for Microsoft 365 Copilot
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner