> For the complete documentation index, see [llms.txt](https://handbook.harmonic.security/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://handbook.harmonic.security/handbook/3.-runtime-sandbox-and-autonomy/3.5-computer-use-desktop-control-risks.md). # 3.5 Computer Use / desktop control risks Screen-and-click control can see and act across desktop apps, creating risks that differ from sandboxed code execution. *Last reviewed: August 18, 2026* {% hint style="info" %} Desktop control gives an agent the user's screen and mouse. That is different from a sandboxed tool call and needs endpoint-level governance. {% endhint %} ## Computer Use / desktop control risks: what security teams need to know Computer Use and desktop-control modes operate through screenshots, clicks, typing, and application context. They can cross from one app into another without the neat boundaries that API tools usually provide. The main control question is whether the organization can see and restrict what the agent can observe or do on the endpoint. Browser and network controls alone usually miss part of this surface. ## Common computer use / desktop control risks security failures * The agent can see secrets or personal data on screen. * A desktop app runs outside browser inspection. * Personal AI accounts are used on managed devices. * Screen-based prompt injection appears inside a document, page, or app. * Endpoint logs do not identify which actions were agent-driven. ## Computer Use / desktop control risks security controls checklist * Restrict desktop-control features to reviewed groups and devices. * Keep password managers, secret stores, HR, finance, and identity admin apps out of agent workflows. * Use endpoint controls to detect desktop app usage and personal-account sign-in. * Require approval before cross-app actions and external writes. * Log app, window class, user, session, approval, and outcome where possible. ## Anthropic ### Overview On the desktop, the Anthropic surfaces are Claude Desktop, Cowork, Claude Code, extensions, browser use, and computer use. Desktop extensions are MCP Bundles (.mcpb): archives containing a local MCP server and a manifest describing its capabilities. Computer use is different: Claude takes screenshots and clicks or types directly in approved applications, with no sandbox between Claude and those apps. Remote Cowork sessions on web or mobile can reach local files, browser state, or computer use only through the Claude Desktop app while the device is online. Computer use is currently a Pro and Max research preview, not a Team or Enterprise capability, so do not describe it as an enterprise control surface until that availability changes. ### Anthropic documentation * [Building Desktop Extensions with MCPB](https://github.com/modelcontextprotocol/mcpb) * [Getting Started with Local MCP Servers on Claude Desktop](https://support.claude.com/en/articles/10949351-getting-started-with-local-mcp-servers-on-claude-desktop) * [When to use desktop and web connectors](https://support.claude.com/en/articles/11725091-when-to-use-desktop-and-web-connectors) * [Let Claude use your computer in Cowork](https://support.claude.com/en/articles/14128542-let-claude-use-your-computer-in-cowork) * [Use Claude Cowork safely](https://support.claude.com/en/articles/13364135-use-claude-cowork-safely) ### Applicable Harmonic guides for Anthropic * [Securing Claude Cowork: A Security Practitioner's Guide](https://www.harmonic.security/resources/securing-claude-cowork-a-security-practitioners-guide) ## OpenAI ### Overview Keep hosted and local OpenAI control separate, because their evidence differs. ChatGPT agent mode is a hosted virtual computer, so its actions do not appear in endpoint telemetry. Local Computer Use runs in ChatGPT Desktop (Work / Codex) on macOS and Windows in both Work and Codex. It can see and operate graphical applications outside the project workspace, so operating-system permissions and the app's allowed-application list become part of the control plane. ChatGPT Desktop (Work / Codex) combines Chat, Work, and Codex in one installation. Review which executable is deployed, whether Computer Use is installed and enabled, which apps are approved, and which mode initiated the action. ### OpenAI documentation * [ChatGPT desktop app](https://learn.chatgpt.com/docs/app) * [Computer Use](https://learn.chatgpt.com/docs/computer-use) * [Moving to ChatGPT Desktop (Work / Codex)](https://help.openai.com/en/articles/20001276) * [Introducing ChatGPT agent: bridging research and action](https://openai.com/index/introducing-chatgpt-agent/) * [ChatGPT desktop app and Codex clients](https://learn.chatgpt.com/docs/features) * [RBAC](https://help.openai.com/en/articles/11750701-rbac) ### Applicable Harmonic guides for OpenAI * [Securing ChatGPT Enterprise Guide](https://www.harmonic.security/resources/securing-chatgpt-enterprise-guide) ## Frequently asked questions about computer use / desktop control risks ### What is Computer Use in AI tools? It is a family of capabilities where the agent observes a screen and acts through clicks, typing, and screenshots rather than structured API calls. It comes in two shapes: hosted virtual computers, such as ChatGPT agent mode, where the desktop belongs to the vendor, and local desktop control, where the agent acts on the user's own machine. The two need different controls and produce different evidence. ### Why is desktop control different from API tools? A screenshot captures whatever is visible, including notifications, password managers, other apps, and personal data, not just what was intentionally shared, and clicks can cross app boundaries that API scopes would have enforced. Screen content is also an injection surface: instructions rendered in a document or page can steer the agent. The tidy per-tool permission model largely disappears. ### Should desktop control be allowed on managed devices? Only for reviewed use cases, restricted user groups, and devices where endpoint telemetry can attribute agent actions. Use platform gates where they exist, such as role-based access for agent capabilities and enterprise controls for desktop extensions, rather than relying on user discipline. Handle personal AI accounts on managed devices first, since they bypass every workspace control. ### What apps should be off limits? Start with password managers, identity admin consoles, finance, HR, healthcare, legal, source-control administration, and production cloud consoles. The pattern: apps where one click is irreversible or one visible screen is a reportable exposure. Keep those apps closed during agent sessions rather than trusting the agent to ignore them. ### How can teams monitor desktop-control risk? Combine endpoint telemetry for which apps ran and which windows were active, AI-platform logs where they exist, app inventory for desktop extensions and local servers, and user reporting. Know the gaps in advance: hosted virtual computers leave no endpoint trace, and local extension activity may not appear in vendor compliance feeds. Section 6.6 maps evidence by surface. ## Applicable regulations and frameworks | Governance page | Relationship to this article | | --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | G.1 Map your controls to NIST AI RMF and CSF 2.0 | This article supplies implementation evidence for the NIST AI RMF and matching NIST CSF 2.0 outcomes. | | G.3 DORA and AI resilience in financial services | Conditional: for a DORA-regulated workflow, this supports ICT containment, resilience, continuity, and recovery. | | G.6 Write an AI Acceptable Use Policy that holds up | This article supplies a technical or process control used to enforce the acceptable-use policy. | | G.7 Ownership and RACI for AI security | This control depends on the ownership and evidence responsibilities defined in the RACI. | *G.2, G.3, G.4, and G.9 are conditional mappings. They apply only when the deployment is within the legal or regulatory scope described on the linked governance page.* ## Related handbook guidance * [3. Runtime, Sandbox & Autonomy](/handbook/3.-runtime-sandbox-and-autonomy.md) * [3.1 What Even is an AI Sandbox?](/handbook/3.-runtime-sandbox-and-autonomy/3.1-what-even-is-an-ai-sandbox.md) * [3.4 Internet access and browser automation](/handbook/3.-runtime-sandbox-and-autonomy/3.4-internet-access-and-browser-automation.md) * [4.6 Cross-app data flow and live artifacts](/handbook/4.-data-protection-and-residency/4.6-cross-app-data-flow-and-live-artifacts.md) * [5.1 Prompt injection: the connective risk](/handbook/5.-threats-and-adversarial/5.1-prompt-injection-the-connective-risk.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://handbook.harmonic.security/handbook/3.-runtime-sandbox-and-autonomy/3.5-computer-use-desktop-control-risks.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.