A cost-effective way to improve SBC performance on AI tasks

by
url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url url

Key findings

  • The Raspberry Pi AI Kit boosts performance with 13 TOPS and is ideal for developers working on object and pose detection tasks.
  • The AI ​​Kit may not be as good at handling LLMs, but it excels at camera-based machine learning tasks like object detection.
  • Not recommended for casual users; better suited for developers who want to dive deep into Python and ML tools.



The Raspberry Pi ecosystem is full of accessories, HATs and sensors that can take the capabilities of your SBC model to the next level. Since the Raspberry Pi 5 adds a PCIe interface to its already robust port selection, you can connect even more attachments to the SBC. One of these is the recently released Raspberry Pi AI Kit, which consists of an M.2 HAT+ and a Hailo-8L NPU.

A Raspberry Pi 5 held in front of a PC

Related

A beginner's guide to programming the Raspberry Pi

Not sure what to do after purchasing your first Raspberry Pi? Check out our detailed tutorial to get familiar with the SBC.

If AI modules for the Raspberry Pi 5 sound familiar, that's because Coral released a $60 USB accelerator a while back. What's special about the official first-party AI kit, however, is that it adds a whopping 13 TOPS of processing capabilities to the RPi 5, compared to the 4 TOPS offered by the Coral kit.


Given these incredible numbers, you might be wondering if there's a catch to the accessory. As it turns out, the Raspberry Pi AI Kit's biggest problem is that it's specifically aimed at developers, rather than the average user who just wants to experiment with LLMs. Don't get me wrong: In the right hands, this inexpensive accessory can be a real game-changer. But for the general public, the device quickly becomes boring after running a handful of object detection demos with it.

A transparent rendering of the Raspberry Pi AI Kit

Raspberry Pi AI Kit

The Raspberry Pi AI Kit bundles an M.2 HAT+ with a Hailo AI acceleration module so you can use the NPU with the Raspberry Pi 5. It provides an accessible, cost-effective and energy-efficient way to integrate powerful AI.

About this article: I purchased the Raspberry Pi AI Kit last month to satisfy the tinkerer in my brain, so the Raspberry Pi Foundation had no influence on the content of this article.



Prices and availability

The Raspberry Pi Foundation released the AI ​​Kit in early June. The device retailed for $70. But as is the fate of any first-party accessory for the hugely popular lineup, you'll have a hard time finding the Raspberry Pi AI Kit as it's running low on stock. At the time of writing, many retailers, including CanaKit, CED, and Vilros, have it listed on their websites. However, you may have to wait a while as it's sold out on most websites. Well, you can technically buy it on Amazon, but you'll have to shell out almost double the launch price of the device.


Setup and installation

Usually relatively painless

The Raspberry Pi AI Kit includes everything you need to mount it on the RPi 5, and the whole process is pretty straightforward. First, you'll need to install the four standoffs on the base of the Raspberry Pi 5 and place the GPIO extender on the pins. Once that's done, you can plug the ribbon cable on the AI ​​Kit's M.2 HAT+ into the Raspberry Pi 5's PCIe socket before securing the accessory onto the standoffs with the final set of screws. Just don't forget to connect a camera module to the SBC afterward.


The terminal command to install the dependencies for the Raspberry Pi AI Kit

Configuring the software aspect is just as easy, but you will need to use the 64-bit version of the Raspberry Pi operating system to use the camera. All you need to do is enable PCIe 3.0 support in the Raspi configuration settings and sudo apt install hailo-all Command to install the required dependencies for the AI ​​Kit.

software

Somewhat inadequate for the average consumer

The terminal command to check if the Raspberry Pi AI Kit is connected to the Raspberry Pi 5


When I first heard about AI Kit, I was keen to test it with some LLMs and compile the benchmarks for most of the popular models. When I tried to run some language models via Ollama on my RPi 5 a few months ago, the results were less than impressive. But as it turns out, AI Kit is not powerful enough to run full-fledged LLMs.

Instead, the Hailo 8L processor is only compatible with machine learning tasks that involve the feed captured by camera modules. And yes, I meant camera modules, not any old webcam or IP camera. That's because the demos available on GitHub only support the RPiCam package, which in turn runs on camera modules. Having tested both my OBSBOT Tail Air and Aicoco onAir webcams in various modes, I can confirm that you need a proper camera module that plugs into the MIPI socket.


Nevertheless, I ran the object detection demo via the Yolov6 interface and the performance was mostly quite good. However, there is still a lot of work to be done to improve the accuracy of the AI ​​Kit. In many cases, the Raspberry Pi was unable to detect all objects in the frame. Even when it displayed a border around an object, it was unable to properly identify said object.


Switching to Yolov8 improved things significantly. While it was by no means perfect, Yolov8 was significantly better at identifying the objects in the frame. I also tested the YoloX model and its performance and identification abilities were somewhere between Yolov8 and Yolov5. Despite Yolov5's problems with object identification, the model performed surprisingly well on the person and face detection test.


Testing the image segmentation capabilities of the Raspberry Pi AI Kit using the Yolov5 model

Image segmentation test with Yolov5

Next, I ran the image segmentation test. Even after repeating the test with different objects, the AI ​​Kit was only able to color one element in the entire frame, so it wasn't as successful as the other tests.


The Pose Estimation test, meanwhile, produced the best results yet, as the Raspberry Pi was able to create a surprisingly accurate wireframe version of my pose.

Aside from these tests, the Hailo-8L NPU is compatible with a handful of other AI models, which you can get from the official GitHub page. While it's good to know that the NPU will integrate support for Picamera2 and CLIP apps, that's pretty much all you can do with the kit at the moment.

Should you buy the Raspberry Pi AI Kit?

The Raspberry Pi AI Kit connected to a camera module

The AI ​​Kit is worth your money if:

  • You are a developer who needs better performance on object and pose detection tasks.
  • You don't mind paying $70 for an AI kit with a limited number of demos.


You should avoid the AI ​​Kit if:

  • You want more performance when running LLMs on your Raspberry Pi.
  • You are not a big fan of artificial intelligence projects and just want to bring some cool, non-AI/ML related ideas to life with your SBC.

From the title, you can tell that this wasn't going to be a review where I dissect every aspect of the AI ​​Kit – as there isn't much to experiment with other than a handful of demos. Personally, I think the AI ​​Kit is worth the $70 I spent. But that's because I have some project ideas that can benefit from the 13 TOPS power of the Hailo-8L. Sure, it might not be able to run LLMs, but the AI ​​Kit's superior pose detection and face recognition capabilities can help with automation projects.


But that's just my perspective as someone who is interested in programming. If you don't want to write Python codes or dive into the vast world of Pytorch and other machine learning tools, the AI ​​Kit is not for you. So I would advise the average user to look around for useful accessories instead of spending their money on the AI ​​Kit.

A Raspberry Pi 5 with Windows 11

Related

The 10 best operating systems for Raspberry Pi 5

Not sure which operating system to use with your Raspberry Pi 5? Here are ten operating systems worth considering for the palm-sized system

Leave a Comment