Raspberry Pi 5 gets LLM smarts with AI HAT+ 2

Raspberry Pi has launched the AI HAT+ 2 with 8 GB of onboard RAM and the Hailo-10H neural network accelerator aimed at local AI computing.

On paper, the specifications look great. The AI HAT+ 2 delivers 40 TOPS (INT4) of inference performance. The Hailo-10H silicon is designed to accelerate large language models (LLMs), vision language models (VLMs), and "other generative AI applications."

Computer vision performance is roughly on a par with the 26 TOPS (INT4) of the previous AI HAT+ model.

These components and 8 GB of onboard RAM should take a load off the hosting Pi, so if you need an AI coprocessor, you don't need to blow through the Pi's memory (although more on that later).

The hardware plugs into the Pi's GPIO connector (we used an 8 GB Pi 5 to try it out) and communicates via the computer's PCIe interface, just like ...