Nabu Casa should sell larger “Home Assistant Green Plus/Max” appliances as mid-range and high-end models with more expansion slots/ports + fast APU/NPU/GPU for onboard AI acceleration to run local LLM

Feature Requests
, , , ,
7

If you already running Home Assistant on a Raspberry Pi or a computer with a spare M.2 slot then the barrier to entry has never been as low as now as you can simply buy either the 26-TOPS Raspberry Pi AI HAT+ or the 13-TOPS Raspberry Pi AI Kit (both available from $70 US-dollar if you can find any in stock) to quickly get started, and since supported in Home Assistant OS 14 should be easy to use.

So you no longer must have expensive GPU if you do not have higher reuirements. If you simply want to experiment with AI agents in Home Assistant to power a conversational agent then you can alternativly even run a local Ollama server for this purpose on fast modern CPU without a dedicated AI accelerator or GPU.

Yes features are still bleeding edge but progress is moving very fast now, just check out JLo’s latest demo in Home Assistant 2024.12 Release Party → https://www.youtube.com/watch?v=9Y9YY_YHNBY

Not ”the goal” but having a on-premises voice assistant () with option to run AI agent locally with local hardware acceleration is certainly one of the goals, even if it is experimatal only at an early stage today:

You can already run a local LLM via Ollama today, and their roadmap does mentions related future plans:

While I can understand you maybe missing all the hype about using an “AI agents” as a conversation agent in Home Assistant if you are personally not interested in voice assistants but if you have read any of the official Home Assistant blog posts or watched some of release party videos (which they dony many demos of both AI agents and local LLM) during the last year or so then you can not have missed some things mentioned about both cloud and local LLM (Large Language Model) running “AI agent” to provide a conversation agent for Home Assistant, at least it has been written and talked about A LOT in Home Assistant blog posts and forum + covered even more by community creators ever since Home Assistant’s year of Voice was first announced in the end of 2022, (especially in concert with a voice assistant), with the latest Home Assistant OS 14 release adding support for Hailo-8 series of AI accelerators.

"On the topic of newly supported hardware, our release of Home Assistant OS 14 will bring support not only for CM5 but also for the Hailo-8 AI accelerator. This is the AI accelerator found in the Raspberry Pi AI Kit or the even more powerful Raspberry Pi AI HAT+ released last month, which is exclusively for the Raspberry Pi 5. For those using a Pi 5 they can now offload AI processing, like object or person detection, to this efficient add-on."

There are also several companies working on various upcoming (or already released) “Edge AI server” that is an seperate appliance which runs Ollama server locally on your network to provide AI acceleration, see example:

and

Anyway, fact is that you can not only already a local Ollama LLM via the Ollama integration but you can run different local LLMs (Large Language Models) via the Ollama integration as its acts like an abstraction API

Anyway, local LLM is becoming more and more with hardware like the Hailo-8 series of AI accelerators (an AI accelerator module for running local LLM faster than you can on a CPU), such as example an Ollama addon for Home Assistant. So you can either create an Ollama server using an Hailo m,odule or Nvidia GPU in a computer on your network or simply put them in the computer running Home Assistant OS with that addon

Also check out this custom component from HACS helping you add an conversation agent using Ollama:

Suggest read these example references for more overview information on how very much on-topic this is:

Again there is loads more coverage about local LLM via Ollama server done by community creators. Ex: