Prompt injection attacks might 'never be properly mitigated' UK NCSC warns

• UK’s NCSC warns prompt injection attacks may never be fully mitigated due to LLM design
• Unlike SQL injection, LLMs lack separation between instructions and data, making them inherently vulnerable
• Developers urged to treat LLMs as “confusable deputies” and design systems that limit compromised outputs

Prompt injection attacks, meaning attempts to manipulate a large language model (LLM) by embedding hidden or malicious instructions inside user-provided content, might never be properly mitigated.

This is according to the UK’s National Cyber Security Centre’s (NCSC) Technical Director for Platforms Research, David C, who published the assessment in a blog assessing the technique. In the article, he argues that many compare prompt injection to SQL injection, which is inaccurate, since the former is fundamentally different and arguably more dangerous.

The key difference between the two is the fact that LLMs don’t enforce any real separation between instructions ...