Computer-Use and TOCTOU: What You Click Is Not What You Get!
Research reproduces a time-of-check/time-of-use flaw in AI agents with computer-use capabilities, showing that UI elements can be swapped between approval and execution to hijack automated actions.
Summary written by editorial AI · Source link below
Last year, Jun Kokatsu disclosed an interesting vulnerability with ChatGPT Operator by exploiting a race condition. I was wondering if I could reproduce this attack chain, and this post describes the results of that research. I had this post drafted for months, and yesterday at the Real-world AI security conference I included a video demo of this attack in my talk and that reminded me that I should finally publish this.
Editorial Analysis
Enterprises piloting autonomous AI agents face a new class of race-condition attacks where approved actions diverge from executed ones — a risk that conventional access controls do not address.
If deploying AI agents with computer-use capabilities, implement server-side action verification rather than relying solely on the agent's visual confirmation of UI state.
AI agents that interact with screens can be tricked by race conditions into performing unintended actions — a growing risk as enterprises automate workflows.
Forward-looking interpretation drafted by editorial AI under human review — not a reproduction of the source. See methodology.
External link — opens at Embrace The Red (AI Security) in a new tab.
More from the AI Security Desk
- Hugging Face warns an autonomous AI agent hacked its network20 Jul
- Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking20 Jul
- Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior20 Jul
- Poison to Detect: Detection of Targeted Overfitting in Federated Learning20 Jul
- Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation20 Jul