Microsoft Says Windows 11's Agentic AI Can Hallucinate

The agents are also vulnerable to cross-prompt injection attacks, during which malicious content embedded in documents or UI elements can override the agent's original instructions.

Microsoft has warned that its agentic AI can make hallucinatory mistakes, behave unpredictably, and become vulnerable to new types of attacks that were not a concern just a year ago. The company's own support documentation acknowledges that AI models "face functional limitations" and warns users only to enable this feature if they understand the security implications.

The company unveiled Agent Workspace in mid-November 2025. The system creates separate Windows sessions in which AI agents operate with their own user accounts and have access to six personal folders: Documents, Downloads, Desktop, Videos, Pictures, and Music. These agents can perform tasks in the background, such as sorting files, converting formats, or extracting information from PDFs, without constant user supervision.

Microsoft warns that these agents are ...