I've been running birdnetpi on a raspberry pi 3 for years. It runs very well on this modest hardware. I also run birdnet-go in a docker container and use my reolink security cameras for the RSTP audio feed to that instance.

There are all sorts of AI/ML models available that can run on smaller hardware. But they are usually very specific and not general purpose, like the LLMs.

I haven't verified it, but it local ML models on audio is pretty efficient and should run fine on a raspberry pi.

It can run on device on your phone https://birdnet.cornell.edu/

Specialized models can be made incredibly efficient. The big tech guys are trying to replace humans which is why their crap takes such huge servers to run. Building local-first/only just means you need to specialize the model and the pipeline to match what you're doing

Audio processing by ai can absolutely happen on a pi.

As an example, whisper.cpp is a high performance speech recognition model, and it does a great job. The biggest version of it as a model requires less than 5GB to live fully in memory, the large model at about 1.5billion parameters, and it works great even CPU only.

The one used most often is small, which has about 244million parameters, and needs about a gig of ram and nothing more.

For a pi 5, usually the base or tiny model are used. Small could be used in most scenarios, but tiny and base are able to run real-time.

So it depends for how local - sometimes its all self contained and can do just fine on a pi, sometimes its local in that it needs an LLM endpoint to hit, but that can be running on another system entirely.

It would really depend on the project for how/where/why for a definition of local.

Listen, everything that they are doing by scorching the planet and turning the tap water brown, could be achieved on CPU or much smaller GPUs on local machines - it's a question of focused, tight training. What they're doing instead is literally attempting to build gigantic models that take in absolutely everything (burning real books in the process because fuck history we got money to make) and attempting to sell the idea that this brute force, one-thing-does-it-all approach, which requires us to give up everything else that is good in the world, is the only way forward, and yes, we must build the roko basilisk now.

It's a big, big grift.

It's a small model that runs on your machine using a TFLite model. AI/ML models come in many different sizes. Some more conducive for running locally than others. There's more info on the detection pipeline here.

I didn't write this, just found and self hosted it.