UAE’s K2 Think AI Jailbroken Through Its Own Transparency Features

Researchers exploited K2 Think’s built-in explainability to dismantle its safety guardrails, raising new questions about whether transparency and security in AI can truly coexist.

K2 Think, the recently launched AI system from the United Arab Emirates built for advanced reasoning, has been jailbroken by exploiting the quality of its own transparency.

Transparency in AI is a quality urged, if not explicitly required, by numerous international regulations and guidelines. The EU AI Act, for example, has specific transparency requirements, including explainability – users must be able to understand how the model has arrived at its conclusion.

In the US, the NIST AI Risk Management Framework emphasizes transparency, explainability, and fairness. Biden’s Executive Order on AI in 2023 directed federal agencies to develop standards including a focus on transparency. Sector-specific requirements such as HIPAA are being interpreted as requiring transparency and non-discriminatory outcomes.

The intent is to protect consumers, prevent bias ...