Researchers poison stolen data to make AI systems return wrong results

Researchers affiliated with universities in China and Singapore have devised a technique to make stolen knowledge graph data useless if incorporated into a GraphRAG AI system without consent.

Large language models (LLMs) base their predictions on training data and cannot respond effectively to queries about other data. The AI industry has dealt with that limitation through a process called retrieval-augmented generation (RAG), which gives LLMs access to external datasets. Google's AI Overviews in Search, for example, use RAG to provide the underlying Gemini model with current, though not necessarily accurate, web data.

GraphRAG represents Microsoft's effort to make RAG more effective. By creating semantically related data clusters called knowledge graphs (KGs), GraphRAG outperforms basic RAG when linked to an LLM-based system. The structuring of the data makes it easier for the LLM to make accurate predictions when prompted.

Amazon, Google, and Microsoft all support GraphRAG in their respective ...