Red teaming LLMs exposes a harsh truth about the AI security arms race

Unrelenting, persistent attacks on frontier models make them fail, with the patterns of failure varying by model and developer. Red teaming shows that it’s not the sophisticated, complex attacks that can bring a model down; it’s the attacker automating continuous, random attempts that will inevitably force a model to fail.

That’s the harsh truth that AI apps and platform builders need to plan for as they build each new release of their products. Betting an entire build-out on a frontier model prone to red team failures due to persistency alone is like building a house on sand. Even with red teaming, frontier LLMs, including those with open weights, are lagging behind adversarial and weaponized AI.

The arms race has already started

Cybercrime costs reached $9.5 trillion in 2024 and forecasts exceed $10.5 trillion for 2025. LLM vulnerabilities contribute to that trajectory. A financial services firm ...