- Home
- AI agents
- What is an AI agent?
- Computer-use agents
Plan · What is an AI agent?
What is a computer-use agent, and when should you use one?
A computer-use agent operates software through its screen: a model looks at screenshots of a browser or desktop, decides where to click and what to type, and a harness carries out each action. Enterprises use it for systems with no API, and it needs more isolation and supervision than an agent that calls APIs.
In this guide: 83 pages
Key takeaways
- Use computer use only where no API, connector or MCP server reaches the system. A tool call avoids sending a screenshot to the model at every step and leaves a cleaner audit trail than a run of clicks.
- Every vendor changed its offer in 2026. OpenAI shut down its computer-use-preview model on 23 July, Google retired the Gemini 2.5 Computer Use model on 28 July and made computer use generally available in Gemini on 30 October, Anthropic took its tool out of beta on 19 August, and Copilot Studio computer use reached general availability in May.
- The screen is an input channel for attackers. Text on a page the agent opens can act as an instruction, which is how researchers turned agentic browsers against their own users in 2025.
- The controls that matter sit outside the model: an isolated virtual machine with no stored credentials, an allow list of sites, confirmation by a person before consequential steps, and a recording of every session.
How does a computer-use agent work?
It runs a loop. The harness, meaning the code around the model, takes a screenshot and sends it with the goal; the model returns the next action, such as a click at given coordinates, typed text or a key press, or a short batch of them; the harness runs the actions and takes the next screenshot, until the task is finished or a person has to step in.
There are two variants. A full computer-use agent drives a desktop inside a virtual machine, so it can work in a legacy Windows application or a terminal emulator. A browser agent drives only a web browser, which narrows what it can reach. Anthropic added a separate browser toolset, for a browser your application hosts, when its computer use left beta in August 2026, and AWS runs a managed, isolated browser as AgentCore Browser.
Each step is a model call with an image attached. Anthropic puts one screenshot at roughly 1,000 to 1,800 tokens, and Copilot Studio bills each computer-use step at 5 Copilot Credits, or 15 with a premium model. A 40-step task can mean dozens of calls before any retries unless the model batches actions, so each step costs more and takes longer than a single API call. Anthropic’s documentation also lists the usual misses: clicks on small targets, and a model that assumes a step succeeded without checking the next screenshot.
Which computer-use agents are available, and what is their status?
Microsoft, AWS and Google all offer computer use, and so do Anthropic and OpenAI through their APIs, but most of these offers changed during 2026. The table gives each one’s status on 2 November 2026. Check again before you build, because two of the first computer-use models no longer exist.
Computer-use and browser agent products and their status as the vendor pages showed on 2 November 2026.
| Product | What it drives | Status on 2 November 2026 |
|---|---|---|
| Copilot Studio computer use (Microsoft) | Websites and desktop applications; a Windows 365 Cloud PC through the Windows 365 for Agents MCP server | Generally available since May 2026; the standalone computer-use tools are still in preview; Windows 365 for Agents generally available since June 2026 |
| Microsoft Foundry Computer Use tool | A browser or desktop; Microsoft says to use virtual machines that hold no sensitive data | Preview terms; runs the limited-access computer-use-preview model, which Azure still documents after OpenAI retired its own copy |
| Amazon Bedrock AgentCore Browser | A managed, isolated browser session | Generally available since 13 October 2025; sessions default to 15 minutes and run up to 8 hours, with Live View, recordings to Amazon S3 and AWS CloudTrail logging |
| Amazon Nova Act | Browser workflows through an SDK and a custom Nova 2 Lite model | Generally available since 2 December 2025 in us-east-1 only; AWS claims over 90% task reliability and publishes no method |
| Gemini computer use (Google) | Browser and screen actions as a built-in tool of Gemini models | Generally available 30 October 2026; the separate Gemini 2.5 Computer Use model was shut down on 28 July 2026 |
| Gemini Enterprise Agent Platform | A Computer Use sandbox and a computer use tool for agents | Sandbox generally available 9 September 2026; tool generally available 30 October 2026 with a human-in-the-loop safety policy |
| Claude computer use (Anthropic) | A desktop in your virtual machine, or a browser your application hosts | Out of beta on 19 August 2026 on the Claude API and Google Cloud; still beta on Amazon Bedrock and Microsoft Foundry |
| OpenAI computer tool | Screens, through the Responses API | The computer-use-preview model shut down on 23 July 2026; the current computer tool shows no GA or beta label on its page |
Two of the first computer-use models, OpenAI’s computer-use-preview and Gemini 2.5 Computer Use, were shut down in July 2026. Put your own action layer between the workflow and the vendor’s tool, so a model change means a configuration change rather than a rebuild.
When should you use a computer-use agent instead of an API?
Use one when the system has no API, connector or MCP server and the work is worth automating anyway, and plan to replace it once an API appears. Anything that can be reached through a tool call is usually cheaper to run that way and easier to control, because no screenshot goes to the model at each step.
Graebel, a relocation services company, has published a case that fits this test. Its Global Connect platform had no API support, so earlier automation proved too rigid, and with Microsoft and a partner it built a Service Order Agent in Copilot Studio that uses computer use to process employee relocation requests across more than 30 service categories. Microsoft reports less manual effort and faster turnaround and gives no figures (Microsoft Copilot Blog, 26 May 2026).
- Mix the two methods in one agent. Read and write through APIs wherever they exist, and use computer use only for the screen that has no endpoint, such as a supplier portal or a legacy order entry form.
- Pick stable, repetitive screens. A form that changes layout every month will cost you retries and reviews.
- Keep a person on the final step of anything that moves money, submits a filing or changes access.
- Stay away from screens that show data the agent has no business seeing, because each screenshot of that screen goes to the model provider.
Against a recorded RPA script, a computer-use agent copes better with a moved button or a renamed field and costs more per run. Where an RPA bot already runs reliably on a stable screen, keep it.
What are the risks of computer-use and browser agents, and how do you control them?
The main risk is that anything on the screen can instruct the agent. A web page, an email or a document the agent opens can carry hidden text that the model treats as a command, and the agent then acts with whatever access its machine and accounts hold.
Researchers showed this on shipping products in 2025. Brave found that hidden text in a Reddit comment made Perplexity’s Comet browser fetch the user’s email address and a one-time password from Gmail and post both back, enough to take over the account; after Perplexity’s fixes, Brave re-tested and found the attack still possible (Brave, 20 August 2025). Radware showed hidden instructions in an email making ChatGPT Deep Research send personal data out through its browsing tool from OpenAI’s own servers, out of sight of enterprise defences (The Hacker News, 20 September 2025). Tenable reported a data exfiltration path through Gemini’s browsing tool, which Google then blocked (Tenable, 30 September 2025).
Vendor documentation says the same. Anthropic runs classifiers on computer-use screenshots and still recommends a minimal-privilege virtual machine, no credentials within reach, domain allow lists and human confirmation for consequential actions. Microsoft’s Foundry page warns of "significant security and privacy risks, including prompt injection attacks". Copilot Studio adds two warnings: an agent shared with maker-provided credentials lets anyone using it act with the author’s access, and its site allow list blocks actions on unlisted sites without stopping the model from opening them.
- Isolation. Run the agent in a dedicated virtual machine or a managed browser such as AgentCore Browser, reset after each session, with no saved passwords, cookies or tokens beyond the task.
- Identity. Give the agent its own account on the target system with the narrowest role, so every action is attributable and the account can be disabled. Never share a maker’s credentials.
- Allow list. Limit the sites and applications it may use, and block downloads and uploads unless the task needs them.
- Confirmation. A person approves the last step of anything that submits, pays, deletes or grants access. Google ships this as a human-in-the-loop safety policy on its computer use tool.
- Recording. Keep the session recording and the action log, and replay a sample every week while the agent is new. AgentCore Browser writes recordings to S3 and its API calls to CloudTrail.
How do Microsoft, AWS and Google implement it?
Where computer use runs on each cloud and the controls each one offers around it, with statuses as the vendor pages showed on 2 November 2026.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Computer use for agents | Copilot Studio computer use (GA May 2026); Foundry Computer Use tool under preview terms | Amazon Nova Act (GA 2 December 2025, us-east-1); Claude computer use on Bedrock (beta) | Gemini computer use (GA 30 October 2026); Agent Platform computer use tool (GA the same day) |
| Isolated environment | Windows 365 for Agents MCP server gives the agent a Cloud PC (GA June 2026) | AgentCore Browser, a managed isolated browser (GA 13 October 2025) | Computer Use sandbox in Agent Platform (GA 9 September 2026) |
| Human confirmation and policy | Multistage approvals in Copilot Studio agent flows (preview); site allow list on computer use | Live View to watch a session; AgentCore Policy on tool calls (GA 3 March 2026) | Human-in-the-loop safety policy on the computer use tool |
| Model lifecycle to watch | Foundry still documents computer-use-preview after OpenAI shut down its own copy on 23 July 2026 | Claude computer use is stable on the Claude API but still beta on Bedrock | Gemini 2.5 Computer Use shut down on 28 July 2026; the built-in tool replaces it |
The same model can carry different terms on different clouds: Anthropic’s computer use is stable on the Claude API and Google Cloud and in beta on Amazon Bedrock and Microsoft Foundry.
Frequently asked questions
Is a computer-use agent the same as RPA?
No. RPA replays a script a developer recorded step by step. A computer-use agent decides each step from a screenshot, so it can handle a layout change that would break a script, and it pays for a model call on every step. Many teams keep RPA for stable high-volume screens and use computer use for the variable ones.
Can a computer-use agent run on an employee’s own laptop?
We advise against it. Whatever is on screen when the agent takes a screenshot, such as an open email or a notification, goes to the model and can carry injected instructions, and the agent acts with the employee’s signed-in sessions. Run it in a dedicated virtual machine or a managed browser, under its own account.
Which model should a computer-use agent use?
Choose from the models your platform supports and test them on your own screens, because vendor reliability claims come without a method. AWS claims over 90% task reliability for Nova Act, and Copilot Studio marks some of its computer-use models as experimental. Pin the model version and retest when it changes.
How do you test a computer-use agent before release?
Record 30 to 50 real tasks on a copy of the target system, including tasks that should stop for a person, and score completion, wrong actions and correct stops. Add pages with planted instructions to see whether the agent follows them. Repeat the run after every model or screen change.
More in What is an AI agent?
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- Computer use in Copilot Studio
- What’s new in Copilot Studio
- New and improved computer-using agents, a new workflows experience and real-time voice experiences
- Computer use tool (Microsoft Foundry Agent Service)
- AgentCore Browser
- Amazon Bedrock AgentCore is now generally available
- Policy for Amazon Bedrock AgentCore is now generally available
- Build reliable AI agents for UI workflow automation with Amazon Nova Act, now generally available
- Gemini API changelog
- Gemini Enterprise Agent Platform release notes
- Claude Platform release notes
- Computer use tool
- OpenAI API deprecations
- Agentic browser security: indirect prompt injection in Perplexity Comet
- ShadowLeak zero-click flaw leaks Gmail data via OpenAI ChatGPT Deep Research agent
- The Trifecta: three new Gemini vulnerabilities in Cloud Assist, Search model and browsing
- Computer use (OpenAI API guide)
- Multistage and AI approvals in Copilot Studio agent flows
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner