ChatGPT Tricked Into Solving CAPTCHAs

AI security platform SPLX has demonstrated that prompt injections can be used to bypass a ChatGPT agent’s built-in policies and convince it to solve CAPTCHAs.

AI agents have guardrails in place to prevent them from solving any CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart), based on ethical, legal, and platform-policy reasons.

When asked directly, a ChatGPT agent refuses to solve a CAPTCHA, but anyone can apparently use misdirection to trick the agent into giving its consent to solve the test, and this is what SPLX demonstrated.

In a regular ChatGPT-4o chat, they told the AI they wanted to solve a list of fake CAPTCHAs and asked it to agree to performing the operation.

“This priming step is crucial to the exploit. By having the LLM affirm that the CAPTCHAs were fake and the plan was acceptable, we increased the odds that the agent would ...