Google's AI Overviews are often so confidently wrong that I’ve lost all trust in them

Have you Googled something recently only to be met with a cute little diamond logo above some magically-appearing words? Google's AI Overview combines Google Gemini's language models (which generate the responses) with Retrieval-Augmented Generation, which pulls the relevant information.

In theory, it's made an incredible product, Google's search engine, even easier and faster to use.

However, because the creation of these summaries is a two-step process, issues can arise when there is a disconnect between the retrieval and the language generation.

While the retrieved information might be accurate, the AI can make erroneous leaps and draw strange conclusions when generating the summary.

That’s led to some famous gaffs, such as when it became the laughing stock of the internet in mid-2024 for recommending glue as a way to make sure cheese wouldn't slide off your homemade pizza ...