BBC probe finds AI chatbots mangle nearly half of news summaries

Four of the most popular AI chatbots routinely serve up inaccurate or misleading news content to users, according to a wide-reaching investigation.

A major study [PDF] led by the BBC on behalf of the European Broadcasting Union (EBU) found that OpenAI's ChatGPT, Microsoft Copilot, Google Gemini, and Perplexity misrepresented news content in almost half of the cases.

An analysis of more than 3,000 responses from the AI assistants found that 45 percent of answers given contained at least one significant issue, 31 percent had serious sourcing problems, and a fifth had "major accuracy issues, including hallucinated details and outdated information."

When accounting for smaller slip-ups, a whopping 81 percent of responses included a mistake of some sort.

Gemini was identified as the worst performer, with researchers identifying "significant issues" in 76 percent of responses it provided – double the error rate of the other AI bots.

The researchers blamed ...