OpenAI bans suspected Chinese accounts using ChatGPT to plan surveillance

OpenAI has banned ChatGPT accounts believed to be linked to Chinese government entities attempting to use AI models to surveil individuals and social media accounts.

In its most recent threat report [PDF] published today, the GenAI giant said that these users usually asked ChatGPT to help design tools for large-scale monitoring and analysis - but stopped short of asking the model to perform the surveillance activities.

"What we saw and banned in those cases was typically threat actors asking ChatGPT to help put together plans or documentation for AI-powered tools, but not then to implement them," Ben Nimmo, principal investigator on OpenAI's Intelligence and Investigations team, told reporters.

One now-banned user, suspected to be using a VPN to access the AI service from China, asked ChatGPT to design promotional materials and project plans for a social media listening tool, described as a "probe," that could scan X, Facebook, Instagram, Reddit ...