Skip to content
  1. Home
  2. AI agents
  3. Platforms and frameworks
  4. Reference architecture on AWS

Build · Platforms and frameworks

What does an enterprise AI agent reference architecture on AWS look like?

This reference architecture is a proposed design for running enterprise AI agents on AWS. It places each part of an agent on a named service: the AgentCore harness or Runtime for the agent, AgentCore Gateway and Policy for tools, AgentCore Identity for access, Bedrock knowledge bases for retrieval and AWS Agent Registry for governance.

Mahesh TatiparthyFounder and CEO, QueryNow
Last reviewed 2 November 2026Product names and statuses as of that date
In this guide: 83 pages

Key takeaways

  1. This is a proposed design assembled from AWS’s documented services; it describes no particular client system. Check each status before you build.
  2. Amazon Bedrock AgentCore is the foundation for new builds. Bedrock Agents Classic closed to new customers on 30 Jul 2026, and AWS points to the AgentCore harness or code-defined agents on AgentCore instead.
  3. Put tool policy at the gateway. AgentCore Policy evaluates Cedar rules on every tool call (GA 3 Mar 2026) and has been able to apply Bedrock Guardrails at that layer since July 2026.
  4. Design for three documented AWS behaviours: knowledge base ACL filtering does not constitute true authorization, the Guardrails prompt attack filter skips tool results, and model invocation logging is off by default.
A proposed reference architecture for enterprise agents on AWSFive layers of a proposed design. Channels: your own applications, Amazon Quick for business users, and Step Functions or schedules for unattended work. Agents: the AgentCore harness for configuration-defined agents, AgentCore Runtime for framework code in an isolated microVM per session, and Strands Agents or LangGraph as the code you own. Tools and knowledge: AgentCore Gateway turning APIs, Lambda functions and MCP servers into tools, AgentCore Policy with Cedar rules on each call, Bedrock managed knowledge bases with ACL-aware and agentic retrieval, and AgentCore Memory. Identity and guardrails: AgentCore Identity, IAM, Bedrock Guardrails and organisation-wide guardrail enforcements. Operate and govern: AgentCore Observability in CloudWatch, AgentCore Evaluations, AWS Agent Registry and CloudTrail.Proposed design, statuses as of 2 Nov 2026ChannelsYour applicationsWeb, mobile, chatAmazon QuickWorkspace for businessusersStep FunctionsUnattended and scheduledrunsAgentsAgentCore harnessAgent from configurationAgentCore RuntimeYour code, microVM persessionStrands or LangGraphFramework code you ownTools andknowledgeAgentCoreGatewayAPIs, Lambda, MCPserversAgentCorePolicyCedar rules percallBedrockknowledge basesManaged, ACL-awareAgentCoreMemoryShort and longtermIdentity andguardrailsAgentCoreIdentityWorkload identity,token vaultIAMSigV4, leastprivilegeBedrockGuardrailsPrompts andresponsesEnforcementsOrganisation-wideguardrailsOperate andgovernAgentCoreObservabilityTraces inCloudWatchAgentCoreEvaluationsOnline and ondemandAWS AgentRegistryApproved agentsand toolsCloudTrailAudit of controlcallsA proposed reference architecture for enterprise agents on AWSFive layers of a proposed design. Channels: your own applications, Amazon Quick for business users, and Step Functions or schedules for unattended work. Agents: the AgentCore harness for configuration-defined agents, AgentCore Runtime for framework code in an isolated microVM per session, and Strands Agents or LangGraph as the code you own. Tools and knowledge: AgentCore Gateway turning APIs, Lambda functions and MCP servers into tools, AgentCore Policy with Cedar rules on each call, Bedrock managed knowledge bases with ACL-aware and agentic retrieval, and AgentCore Memory. Identity and guardrails: AgentCore Identity, IAM, Bedrock Guardrails and organisation-wide guardrail enforcements. Operate and govern: AgentCore Observability in CloudWatch, AgentCore Evaluations, AWS Agent Registry and CloudTrail.Proposed design, statuses as of 2 Nov 2026ChannelsYour applicationsWeb, mobile, chatAmazon QuickWorkspace for businessusersStep FunctionsUnattended andscheduled runsAgentsAgentCore harnessAgent fromconfigurationAgentCore RuntimeYour code, microVM persessionStrands orLangGraphFramework code you ownTools and knowledgeAgentCore GatewayAPIs, Lambda, MCPserversAgentCore PolicyCedar rules per callBedrock knowledgebasesManaged, ACL-awareAgentCore MemoryShort and long termIdentity and guardrailsAgentCore IdentityWorkload identity,token vaultIAMSigV4, least privilegeBedrock GuardrailsPrompts and responsesEnforcementsOrganisation-wideguardrailsOperate and governAgentCoreObservabilityTraces in CloudWatchAgentCoreEvaluationsOnline and on demandAWS Agent RegistryApproved agents andtoolsCloudTrailAudit of control calls
The proposed design in five layers. Each box is an AWS service named on this page; the statuses in the tables below are as AWS’s pages showed them on 2 Nov 2026.

Which AWS service does each part of the agent run on?

Each part of the agent maps to one or two services, most of them in the AgentCore family. The table lists the service we would place there, its status on 2 Nov 2026 and the reason for the choice.

The proposed design layer by layer. Statuses as AWS’s pages showed them on 2 Nov 2026.

PartServiceStatusWhy here
AgentAgentCore harness for agents defined in configuration; AgentCore Runtime for framework codeAgentCore GA 13 Oct 2025; harness GA 17 Jun 2026An isolated microVM per session, any framework or model
FrameworkStrands Agents, LangGraph or another framework on RuntimeStrands 1.0 on 15 Jul 2025Code you own; Strands also powers the harness
OrchestrationStep Functions with an InvokeHarness state; Lambda durable functionsLambda durable functions launched 2 Dec 2025Fixed steps, approvals and long waits outside the agent
ToolsAgentCore Gateway over APIs, Lambda functions and MCP servers; the AWS MCP Server for AWS APIsGateway GA 13 Oct 2025; AWS MCP Server GA 6 May 2026One MCP endpoint, inbound and outbound auth, semantic tool search
Tool policyAgentCore Policy with Cedar rulesGA 3 Mar 2026; Guardrails at the gateway since Jul 2026Decisions outside agent code, with a log-only mode for testing
KnowledgeBedrock managed knowledge bases with ACL-aware and agentic retrievalAgentic retrieval on managed knowledge bases since 2 Mar 2026; docs show no preview labelLive access checks on seven connectors that fail closed
MemoryAgentCore MemoryGA 13 Oct 2025Per-actor namespaces, record TTLs, customer-managed keys
IdentityAgentCore Identity; IAMGA 13 Oct 2025; on-behalf-of token exchange added Apr 2026A workload identity per agent, a token vault, user-bound tokens
GuardrailsBedrock Guardrails, enforced across accounts through AWS OrganizationsGA; Automated Reasoning checks GA 6 Aug 2025Content, PII and prompt attack filters on model calls
Observability and evaluationAgentCore Observability in CloudWatch; AgentCore EvaluationsObservability GA 13 Oct 2025; Evaluations GA 31 Mar 2026Traces, sessions and tokens, plus 13 built-in evaluators
GovernanceAWS Agent Registry; CloudTrailRegistry GA 31 Aug 2026Only approved agents and tools are discoverable by agents

Users reach the agents through your own web, mobile or chat applications, or through Amazon Quick, the workspace AWS now offers business users in place of Amazon Q Business, which is closed to new customers. Unattended agents start from Step Functions or a schedule and run with no user behind them, so they need their own narrow grants.

How does one request move through the architecture?

From your application to the agent with the user’s token, out through the gateway and the knowledge base under that identity, and back with traces in CloudWatch. The numbered path follows one request to an agent on AgentCore Runtime.

  1. A user signs in to your application, which sends the user’s JWT with the request. AgentCore Runtime and Gateway accept inbound JWTs from any OIDC identity provider, checked against allowed audiences, clients and scopes.
  2. The agent runs in AgentCore Runtime, one microVM per session. AgentCore Identity turns the user’s JWT into an agent token bound to that user, so downstream calls carry both identities.
  3. For knowledge, the agent queries a Bedrock managed knowledge base with the user’s verified email in userContext; seven connectors re-check access at the source, and the check fails closed.
  4. For actions, the agent calls AgentCore Gateway. AgentCore Policy evaluates its Cedar rules on the call, and the gateway takes the target credential from an outbound credential provider. AWS recommends on-behalf-of token exchange over token passthrough in production.
  5. Model calls pass through Bedrock Guardrails, which can be enforced organisation-wide through a BEDROCK_POLICY in AWS Organizations; where organisation, account and request guardrails all apply, the most restrictive wins.
  6. A step that needs a person runs in Step Functions, paused on a .waitForTaskToken callback, or in a Lambda durable function with a callback, so the wait holds no compute.
  7. AgentCore Observability writes spans, metrics and logs to CloudWatch, CloudTrail records Gateway and Agent Registry calls, and AWS Agent Registry holds the approved record through which other agents discover this one.

Which AWS behaviours change the design?

Five documented behaviours need a design answer before go-live: retrieval filtering that AWS says does not constitute true authorization, guardrail filters that skip tool traffic, logging that starts switched off, processing that can leave a Region, and gaps in the harness for some multi-agent patterns.

  • Knowledge base ACLs. AWS states that managed knowledge base filtering does not constitute true authorization. Your application must authenticate users and pass a verified email, and a mismatched email fails silently with no results.
  • Tool traffic. The Guardrails prompt attack filter does not evaluate toolResult content or tool definitions, and PII masking does not reach model invocation logs or tool-call arguments and results. Inspect tool arguments and results explicitly at the gateway with AgentCore Policy, which can also apply Guardrails there, and test that coverage with real payloads before relying on it.
  • Logging. Bedrock model invocation logging is disabled by default and covers only the bedrock-runtime endpoint. Turn it on deliberately, decide where bodies over 100 KB land in S3, and set a retention period.
  • Residency. Geographic cross-Region inference profiles such as us, eu, apac and in keep processing within that geography, while global profiles route to any commercial Region. Enforce the choice with service control policies and the bedrock:InferenceProfileArn condition, and check each model’s data retention mode, because some newer models require aws_review.
  • Migration and multi-agent work. Bedrock Agents Classic has a frozen model catalogue, and AWS’s own comparison says routing-mode multi-agent is not straightforward on the harness; full multi-agent collaboration requires custom framework code on Runtime.

In what order should you build it?

Identity and the gateway first, because every later piece depends on them, then one agent end to end, then the controls that let you add the second agent without starting over.

  1. Set up AgentCore Identity with your identity provider, the inbound JWT authorizer and the outbound credential providers, and decide which agents act for users and which act as themselves.
  2. Stand up AgentCore Gateway with AgentCore Policy in log-only mode, put your first APIs behind it, and switch Policy to enforce once the logs look right.
  3. Build one agent on the harness or on Runtime, with a knowledge base and memory only if the task needs them.
  4. Turn on AgentCore Observability, model invocation logging and an AgentCore Evaluations suite built from real cases before anyone outside the team uses the agent.
  5. Register the agent and its tools in AWS Agent Registry and require approval before the next agent can discover them.

How do Microsoft, AWS and Google implement it?

The same layers on the other two clouds, for teams that run on more than one. The Azure design has its own page in this guide.

Capability, Microsoft, AWS, Google

CapabilityMicrosoftAWSGoogle
Agent runtimeFoundry Agent Service; Copilot StudioAgentCore harness and AgentCore RuntimeAgent Runtime in Gemini Enterprise Agent Platform
Agent identityEntra Agent IDAgentCore IdentityAgent Identity (API preview since 18 Jun 2026)
Tool gateway and policyFoundry Toolbox; API ManagementAgentCore Gateway with AgentCore PolicyAgent Gateway with Model Armor
RegistryAgent 365 registryAWS Agent RegistryAgent Registry

Microsoft Agent 365 can sync registry records with Amazon Bedrock in public preview, and Microsoft Defender discovers agents on Amazon Bedrock, so a mixed estate should name one registry as the system of record.

Frequently asked questions

Should new builds use Bedrock Agents Classic?

No. Amazon Bedrock Agents became Bedrock Agents Classic and closed to new customers on 30 Jul 2026, with no new features and a frozen model catalogue. AWS recommends the AgentCore harness, or code-defined agents on AgentCore, for new work.

Is the AgentCore harness enough, or do we need a framework?

The harness covers a single agent with tools, memory and identity defined in configuration. AWS’s comparison says routing-mode multi-agent is not straightforward on it and full multi-agent collaboration needs custom framework code, so teams that need those patterns run Strands or another framework on AgentCore Runtime.

How do agents get credentials for third-party systems on AWS?

Through AgentCore Identity’s outbound credential providers. Its token vault, encrypted with a customer-managed or service-managed KMS key, holds OAuth tokens and API keys, with built-in providers for Google, GitHub, Slack, Salesforce and Atlassian, and the gateway uses them on each call.

Is this an AWS-endorsed architecture?

No. It is our proposed design built from AWS’s documented services, with statuses as of 2 Nov 2026, and it describes no client deployment. AWS publishes guidance per service; check each page before you build, and expect changes after AWS re:Invent at the end of 2026.

More in Platforms and frameworks

Copilot Studio vs Microsoft FoundryRead Microsoft vs AWS vs GoogleRead Agent frameworks comparedRead Build vs buyRead Reference architecture on AzureRead

Sources

Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.

  1. What is Amazon Bedrock AgentCore? AWS documentation, undated page, read 2 November 2026
  2. Amazon Bedrock AgentCore is now generally available AWS What’s New, 13 October 2025
  3. AgentCore Runtime AWS documentation, undated page, read 2 November 2026
  4. AgentCore harness AWS documentation, undated page, read 2 November 2026
  5. Top announcements of the AWS Summit in New York 2026 AWS News Blog, 17 June 2026
  6. Amazon Bedrock Agents Classic maintenance mode AWS documentation, undated page, read 2 November 2026
  7. Amazon Bedrock AgentCore Gateway AWS documentation, undated page, read 2 November 2026
  8. Gateway inbound authorization AWS documentation, undated page, read 2 November 2026
  9. Policy for Amazon Bedrock AgentCore is now generally available AWS What’s New, 3 March 2026
  10. AgentCore release notes AWS documentation, undated page, read 2 November 2026
  11. AWS MCP Server is now generally available AWS What’s New, 6 May 2026
  12. AgentCore Identity AWS documentation, undated page, read 2 November 2026
  13. Securing AI agents with Amazon Bedrock AgentCore Identity AWS Security Blog, 14 October 2025
  14. On-behalf-of token exchange AWS documentation, undated page, read 2 November 2026
  15. AgentCore Memory AWS documentation, undated page, read 2 November 2026
  16. Managed and customer-managed knowledge bases AWS documentation, undated page, read 2 November 2026
  17. ACL-aware retrieval in managed knowledge bases AWS documentation, undated page, read 2 November 2026
  18. Amazon Bedrock document history AWS documentation, undated page, read 2 November 2026
  19. Amazon Bedrock Guardrails AWS documentation, undated page, read 2 November 2026
  20. Prompt attack filters AWS documentation, undated page, read 2 November 2026
  21. Bedrock Guardrails sensitive information filters AWS documentation, undated page, read 2 November 2026
  22. Guardrails enforcements AWS documentation, undated page, read 2 November 2026
  23. Automated Reasoning checks in Amazon Bedrock Guardrails generally available AWS What’s New, 6 August 2025
  24. AgentCore Observability AWS documentation, undated page, read 2 November 2026
  25. AgentCore Evaluations generally available AWS What’s New, 31 March 2026
  26. Model invocation logging AWS documentation, undated page, read 2 November 2026
  27. AWS Agent Registry now generally available AWS What’s New, 31 August 2026
  28. Cross-Region inference AWS documentation, undated page, read 2 November 2026
  29. Data retention in Amazon Bedrock AWS documentation, undated page, read 2 November 2026
  30. Choosing workflow type in Step Functions AWS documentation, undated page, read 2 November 2026
  31. Build multi-step applications and AI workflows with AWS Lambda durable functions AWS News Blog, 2 December 2025
  32. Introducing Strands Agents 1.0 AWS Open Source Blog, 15 July 2025
  33. Amazon Q Business availability change AWS documentation, undated page, read 2 November 2026

Planning an AI program?

Talk to the team that builds it and runs it after launch.

Talk to our team See our workTell us the workflow
  • Built to SOC 2, HIPAA and GDPR standards
  • EU AI Act-aligned delivery
  • Microsoft Solutions Partner