Nabu Casa should also sell larger “Home Assistant Green Plus” appliances as mid-range/high-end models with more expansion slots/ports and fast CPU/NPU/GPU for onboard AI acceleration

Feature Requests
, , , ,
1

Not seen any hardware roadmaps mention faster applicance hardware to run Home Assistant OS on so wondering if Nabu Casa could make/sell some larger Home Assistant OS appliance models (mid-range variant) in the Home Assistant Green series with maybe more M.2 slots + more USB-ports for expansion and faster SoC/CPU with onboard native capabilities to run local AI acceleration via embedded or optional NPU/GPU?

That is, since the existing “Home Assistant Green” model has very low-level hardware specifications but seems to be a huge success why not also make and sell one or more faster/larger models with more on-board expansion options to target those that want the same simple plug-and-play appliance experience but know they want to add both AI accelerator M.2 card (like Hailo Accelerators) and an M.2 NVMe storage (so need SoC with more PCIe lanes or PCIe switch) + want to add more USB radio dongles but still wanting an easy to use all-in-one hardware solution that it simple to get started with and maintain?

Why should Nabu Casa not design a larger model of Home Assistant OS appliance in the same series called “Home Assistant Green Plus” or something similar that offer more PCIe expansion options out-of-the-box to enable optional PCI Express (PCIe) hardware acceleration modules/cards to be added (such example the one that comes with the Raspberry Pi AI Kit), and then continue to offer the standard “Home Assistant Green” as the low-level model (more commonly referred to as entry-level model).

I’m hoping to see a mid-range model with not only more expansion options but also better SoC for AI:

  • Two long M.2 PCIe slots that can be used for combo of AI accelerator module and NVMe SSD disk.
  • Four USB 2.0 ports for external USB dongles (Z-Wave, Zigbee, Thread + Bluetooth radio adapters).
  • One SD-card slot for optional local backups.

Alternativly if do not want add two M.2 PCIe slots to the mid-range model then consider adding a dedicated AI accelerator IC directly to the board (such as example the Hailo-8 that is used on the Raspberry Pi AI HAT+).

Maybe consider adding PoE (Power-over-Ethernet) as well if want to retire the Home Assistant Yellow(?).

If that works out then perhaps after a year or two consider also designing an even larger high-end model to be the new flagship variant in the same series called “Home Assistant Green Max” or similar with even more M.2 slots and USB-ports for further expansion.

  • At least one but preferably two short M.2 PCIe slots for AI accelerators (like Hailo Accelerators).
  • Two or more M.2 slots for NVMe SSD disks (with option to configure RAID-1/mirror for redundancy).
  • More than four USB 2.0 ports for external USB dongles like radio adapters.
  • One SD-card slot for optional local backups.

By the way, another argument for consolidating and combining an AI accelerator together with many NVMe SSD disks in a single Home Assistant appliance is use case for running an NVR (Network Video Recorder) like Frigate as an add-on in Home Assistant for both locally processed AI and video recording. For that reason alone it would be great to have more than two NVMe slots (at a lower speed) instead only two at higher speeds.

Anyway, one of the main point is that with AI acceleration becoming interesting for more people and new hardware acceleration options being made available as M.2 models so fast it would be nice if there were more options to slowly grow with AI acceleration, by booth first starting out with onboard AI acceleration capabilities embedded into the SoC/CPU and then having various options to add different M.2 PCIe adapter as optional modules to add more AI capabilities to your existing appliance.

While it is not nessesarly a must it would also be nice for higher-end models to use an “AI-ready” SoC/CPU/chipset so it also has one or more fast embedded NPUs (for AI acceleration onboard out-of-the-box), so if ever make even higher higher-end models in the future could worth it to look into basing such models on an x86-64 CPU/chipset from Intel or AMD instead of an ARM SoC, (such as example the new AMD Ryzen AI 300 series and Intel Core Ultra 200V series which are designed to meet the NPU AI TOPS performance requirements of Microsoft Copilot+ / Copilot Plus laptops PCs).

Note! Yes I know there Nabu Casa still sells the Home Assistant Yellow model but it only has one M.2 slot and because it is currently only sold as a kit today it is not plug-and-play so do not work out-of-the-box.And yes, I also know that at the price we are talking here it would probably make more economical sense for end-users wanting those features to just buy a mini-PC instead, but that negates the idea of “Home Assistant Green” series as complete and finished appliance products that are all easy for new/beginner users to get started with and simple to maintain.

PS: Off-topic but a related idea it might also be the concept of doing a free open-source hardware project for the community in parallell to cater to all the thinkers too by helping the community designing a brand new carrier-board for the new Raspberry Pi Compute Module 5 (as a spiritual successor of the Home Assistant Yellow but redesigned for the CM5 and with leasons learned) and release that new PCB design schematics as an updated reference design (+ an enclosure for 3D-printing) that can be shared under projects/marketplaces for DIY PCB manufacturers sites like PCBWay and JLCMC (which if the open-source hardware license allows should even allow third-parties to build such offerings?)? → GitHub - NabuCasa/yellow: Home Assistant Yellow - hardware for Home Assistant which grows with your needs. …I guess today much of such a design could also be learned from the new Raspberry Pi Compute Module 5 IO board which they published Kicad design files for now:

2

They might do a refresh of the current ones with these new options when its time and as more people request them. It takes time to make sample and development versions then make them viable for market.

3

Well they should probably hurry up now if they want more users to run an on-premises AI agent with their upcoming voice hardware that is planned to be released next week :wink: see → https://youtube.com/live/ZgoaoTpIhm8

In case you missed it, “the biggest announcement of the year has yet to come though” as Nabu Casa will launch some kind of first-generation Home Assistant Assist Satellite voice smart speaker hardware product (based on XMOS XU316 DSP chip for advanced audio processing + ESP32-S3 running ESPHome) during the the live stream of Voice: “What’s new in voice - Chapter 8” on the 19th of December 2024 (which now state “Join us as we announce the voice assistant hardware we’ve been working hard on all year and see what’s new since our last Voice chapter!”)

They mentioned during the last Home Assistant release party that it will have immediate availability and that they have already manufactured A LOT of them so should not be a problem for loads of users to get it. A lot more info purposely “leaked” about that voice hardware in the Home Assistant 2024.12 release notes:

Regardless, I think the existing Home Assistant Green hardware specifications do not need a refresh for it to continue as an entry-level model for users who do not want or need on-premises AI or more local storage.

4

What makes you think that is an inherent goal of the soon-to-be-released hardware?

5

Hi Hedda,

I believe because things like a Beelink are so cheap, you can’t compete with it, so why try?

The green filled a niche as a preloaded entry level model that is supposed to just work.

Also, often times you can say more with fewer words, because people will actually read it all. That’s a LOT of word salad there.

3 Likes
6

As far as I know local AI (in the sense of LLMs to control voice assistants) is still a bleeding edge use case that is both expensive, unreliable and difficult for non-enthusiasts.

Top performing models that can reliably control a smart home need either a beefy GPU that is at least several hundred dollars, or a lot of RAM with expensive CPU like Mac M3. The Copilot-ready HW doesn’t have support for Ollama at the moment from what I can tell, and it’s unclear how much NPUs will help, in my understanding the main specs needed are memory bandwidth and capacity, not compute power. Because of the current AI craze, anything which can run LLMs is expensive.

Even when it does work, it’s a lot more power hungry and chaotic compared to the current intent handling. You are at the whim of the LLM and your prompt to decide what it should do, and it is very difficult to provide a good user experience using local models, even top models like Sonnet 3.5 of GPT4 get it wrong sometimes.

Given the current state I think it is too early for Nabu Casa to go into local AI. Hopefully the tech matures quickly and we can consider it soon but IMO it is still in the early adopter/enthusiast phase for now.

1 Like
7

If you already running Home Assistant on a Raspberry Pi or a computer with a spare M.2 slot then the barrier to entry has never been as low as now as you can simply buy either the 26-TOPS Raspberry Pi AI HAT+ or the 13-TOPS Raspberry Pi AI Kit (both available from $70 US-dollar if you can find any in stock) to quickly get started, and since supported in Home Assistant OS 14 should be easy to use.

So you no longer must have expensive GPU if you do not have higher reuirements. If you simply want to experiment with AI agents in Home Assistant to power a conversational agent then you can alternativly even run a local Ollama server for this purpose on fast modern CPU without a dedicated AI accelerator or GPU.

Yes features are still bleeding edge but progress is moving very fast now, just check out JLo’s latest demo in Home Assistant 2024.12 Release Party → https://www.youtube.com/watch?v=9Y9YY_YHNBY

Not ”the goal” but having a on-premises voice assistant () with option to run AI agent locally with local hardware acceleration is certainly one of the goals, even if it is experimatal only at an early stage today:

You can already run a local LLM via Ollama today, and their roadmap does mentions related future plans:

While I can understand you maybe missing all the hype about using an “AI agents” as a conversation agent in Home Assistant if you are personally not interested in voice assistants but if you have read any of the official Home Assistant blog posts or watched some of release party videos (which they dony many demos of both AI agents and local LLM) during the last year or so then you can not have missed some things mentioned about both cloud and local LLM (Large Language Model) running “AI agent” to provide a conversation agent for Home Assistant, at least it has been written and talked about A LOT in Home Assistant blog posts and forum + covered even more by community creators ever since Home Assistant’s year of Voice was first announced in the end of 2022, (especially in concert with a voice assistant), with the latest Home Assistant OS 14 release adding support for Hailo-8 series of AI accelerators.

"On the topic of newly supported hardware, our release of Home Assistant OS 14 will bring support not only for CM5 but also for the Hailo-8 AI accelerator. This is the AI accelerator found in the Raspberry Pi AI Kit or the even more powerful Raspberry Pi AI HAT+ released last month, which is exclusively for the Raspberry Pi 5. For those using a Pi 5 they can now offload AI processing, like object or person detection, to this efficient add-on."

There are also several companies working on various upcoming (or already released) “Edge AI server” that is an seperate appliance which runs Ollama server locally on your network to provide AI acceleration, see example:

and

Anyway, fact is that you can not only already a local Ollama LLM via the Ollama integration but you can run different local LLMs (Large Language Models) via the Ollama integration as its acts like an abstraction API

Anyway, local LLM is becoming more and more with hardware like the Hailo-8 series of AI accelerators (an AI accelerator module for running local LLM faster than you can on a CPU), such as example an Ollama addon for Home Assistant. So you can either create an Ollama server using an Hailo m,odule or Nvidia GPU in a computer on your network or simply put them in the computer running Home Assistant OS with that addon

Also check out this custom component from HACS helping you add an conversation agent using Ollama:

Suggest read these example references for more overview information on how very much on-topic this is:

Again there is loads more coverage about local LLM via Ollama server done by community creators. Ex:

8

Thought I had indirectly explained my reasoning in my original post above that it is fact that many (but not all) people simply do not want run Home Assistant on DIY hardware even if such solution can often be simple to put together their own from off-the-shelf parts, instead many just want buy a complete and finished product because it is plug-and-play and designed to work out-of-the-box with as little time/effort to get started and maintain as possible.

If this was not the case then Nabu Casa would not have sold any Home Assistant Blue and Home Assistant Yellow as people could instead at that time buy Hardkernel’s ODROID-N2/ODROID-N2+ or a Raspberry Pi 4 for much less money. I think many existing users (like myself) buy stuff like that because we not only want a finished product but there are also other non-logical reasons (such as buying official Home Assistant branded products because of brand loyalty and believing it will a way directly and indirectly support this project), or simply for bragging rights and showing off to friends :wink:

But let me to add some better comparison from my perceived market and target audience point-of-view to other types of some shat similar hardware prodocts for which many people buy way more costly pre-built products that users could relatively easily put together alternative DIY-style solutions for much less cost but many users instead choose to buy a finished all-in-one product for much more money:

  • NAS (Network-Attached Storage) - Many if not most people to get a NAS who want the function but not the hassle of DIY:ing the hardware (like myself) buy expensive finished product from Synology, Qnap, Asustor Terramaster, (and now also Ubiquiti), etc. instead of just putting together a DIY solution from off-the-shelf computer parts (yes I know you can even buy inexpensive complete NAS-like chassis) using free firmware/operatingsystem/software such as TrueNAS, Unraid, OpenMediaVault, XigmaNAS (formerly NAS4Free), EasyNAS, SnapRAID etc…

  • NVR (Network Video Recorder + Camera Video Management) - Again, many if not most people choose to get either an all-in-one dedicated product (like example Reolink NVR) or use the NVR solution provided in their NAS (like Synology) or Router (like Ubiquiti’s Unifi Protect) instead of using DIY solutions such as example Frigate or Blue Iris.

  • Router/Firewall (and Wireless router) - Think one might not be the most obvious but I bet that most of us here have probably just bought finished Wi-Fi Router/Firewall product(s) from Asus, Netgear, D-Link, Ubiquiti (Unify), or Synology instead going with an alternative DIY solution when it today can actually be relativly easy to put together your own DIY Router/Firewall and Wireless router solution using off-the-shelf (ARM or x86 based) hardware parts and operatingsystem/software such as example OpenWRT, DD-WRT, Tomato/FreshTomato, pfSense, OPNsense, Zenarmor, or similar.

Everything about Home Assistant is niche. I believe that mid-range and high-end all-in-one models would fill two more niches as some people will prefer to buy a completey finished product instead of putting together a DIY solution, and some of those prefer to buy one brand because of brand loyalty or other reasons. Whatever the reason

This is partially also why I bought branded merchandise from the official Home Assistant Store (and branded products from other similar projects) to show my support for things I like. I guess could instead just always black black polo shirt and blue jeans like Steve Jobs, but I do not want to :stuck_out_tongue:

PS: Speaking for myself, my day-job is a server and storage sysadmin so I could even get old but free enterprise hardware if I wanted but instead for our home I choose to buy a Home Assistant Blue, Home Assistant Yellow, a Synology NAS, and Ubiquiti Unify network components instead of putting together DIY solution from off-the-shelf, (and partily also that I do not want to have another job maintaining several DIY solutions at home when I do similar stuff at work even if those are enterprise solutions).

I know that rant on but you do not need to read it, it is part of my own personal process, I have OCD and I prefer to research stuff myself in detail rather than waiting for someone to spoonfeed me details bit by bit. Still do not think there is an excuse for you posting abusive personal attack comments that are not only off-topic but offensive.

9

I have to agree about encouraging a higher-end plug and play solution. I’m kind of locked in FOBO before getting started. Given how much time and money one has to spend for the rest of the components required to smarten up a home, I would be very surprised if a $/£/€ 300 ready to go device that is fairly well future proofed didn’t sell better than the Green.

1 Like
10

If you already running Home Assistant on a Raspberry Pi or a computer with a spare M.2 slot then the barrier to entry has never been as low as now as you can simply buy either the 26-TOPS Raspberry Pi AI HAT+ or the 13-TOPS Raspberry Pi AI Kit (available from $70 US-dollar) to quickly get started.

So you no longer need expensive GPU, and you if you simply want to experiment with it you can even run a local Ollama server for this purpose on modern CPU without a dedicated AI accelerator or GPU.

These AI accelerators do not work for LLMs, they are meant for other purposes (normally computer vision but not an expert in this area). At the moment you need a GPU with plenty of VRAM if you want an correct answer in short time. If you want assist integration, most of the smaller 3B/7B I have tried do not work well. You can also use a CPU with a lot of RAM, but then it is slow for a lot of answers.

Basically at the moment, you can have either cheap, fast, high quality, but only choose 2. Even then, LLMs still do strange things, and are slower than the inbuilt intent handler.

11

Ah, apparently the less expensive modules in the Hailo family will not work unless the LLM model has specifically using thier compiler on very powerful hardware.

But they instead have a newer and more expensive Hailo 10 series with a Hailo 10H M.2 model that might work with Ollama!

12

I’m still not hopeful, from the looks of this thread on reddit it’s only 8GB which is pretty small for function calling.

13

Ollama smallest model (llama 2) is 7B parameters and looks to require at least 8GB of RAM so guess similar for these AI accelerators or?

However that is out-of-the-box but I understand it is possible to finetune those models using several faster GPUs first (locally on another computer or more likely in the cloud) which can reduce the RAM requirements?

14

Llama 3.1 has an 8B model which I have used on my N100 machine, but they are still pretty terrible for assist. It’s definitely OK if a bit slow for a fallback, but not usable as a main model.

From what I understand fine tuning will not reduce the model size but can squeeze better results for specific use cases from smaller models.

15

There were too many words to read, and you spent too much getting your AI to write them all for you, IMO.

That AI hat thing is probably already out of date, that’s why HA isn’t wasting resources on a tool for that, yet.

Teach a man to say things in fewer words so that people can read them and you end up spreading the word faster.

3 Likes
16

@Sir_Goodenough Do you realize that most of your comments are not only off-topic but extremly rude and offensive, they are a personal attack that have absolutly nothing to do with the topic at hand, they only make it seem like you are trolling. For the record I have never used AI to write a single word here (if I did had then I am sure it would be better written). Yes I know I use too many words, however the reason why is I have ADHD and OCD as well as probably being on the Autism Spectum, and explaining things in details is my coping mechanism for making myself understod, and on top of that English is not my first language, so those are my excuses. What is your excuse for trolling and being a jerk doing so? In my opinion the unnecessary negativity and unrelated acusations in your posts is certainly not contributing anything constructive to this discussion.

17

Lets not degrade things into back and forth attacks.

1 Like
18

Hedda, his words aren’t really offensive, they are more in line with constructive criticism. I believe you are searching for negative inflections in his post. FYI

Yes please.

2 Likes
19

No offense meant. I just found the massive amount of words that take forever to get to the point a bit offensive myself, so I decided to point that out. My OCD flavor will not let me read all of that, and my cognitive memory disorder gets me frustrated trying to follow the long and involved points.

I appreciate you are contributing to the community, but perhaps starting a wordpress blog and posting links to it with TL:DR shorter posts here would allow you to add even more detail in your dissertations and the links from here would be the connection to the HA world to get your point out there. Then people interested in what you say could read it. You have full editorial control. And you don’t have to put up with Jerks like me (paraphrasing you a bit). As it is the long an detailed stuff is impossible for me to read with my disabilities and I take offense.

That said, I’m not going to respond any more, I don’t have any intention for this to become destructive rather than constructive.
I wish you well in your endeavors.

20

Back on-topic; NVIDIA just launched their NVIDIA Jetson Orin Nano Super Developer Kit for $249(US) which include a Jetson Orin Nano compute module and a carrier board for it:

image
image1536×864 70.1 KB

At the announcent NVIDIA’s CEO where marketing this kit as ”The World’s Most Affordable Generative AI Computer”:

That specific development kit is based on the 8GB variant model of new NVIDIA Jetson Orin Nano module series which could potentially be good choice for use in a such suggested mid-range model with buitl-in AI acceleration capabilities to run a smaller LLM.

NVIDIA also makes a 4GB variant, two 8GB variants (with different memory bandwidth), and a 16GB variant in that same NVIDIA Jetson Orin Nano series, but the other modules looks to not yet be available or there is at least no price for the other variants that I could find, however I believe a compute module like that 16GB model could potentially be good enough to be used in a high-end model.

image
image1536×864 71 KB

As a proof-of-concept the same NVIDIA Jetson Orin Nano module is used in reComputer J3011 (Edge AI Computer with NVIDIA Jetson Orin Nano 8GB) sold for $599(US) by Seeed Studio: