And will it run on a 2GB Pi 4?
Not on your life… My life or any…
Tensor math is a thing.
In this regard, I am a luddite. Keep it as a user choice.
Just to lean the other way for a moment - why do you need an “Official” AI bot.
It’s fairly easy to expose the config directory via Samba and you can generate/provide a HA API key to the AI of your choice.
TL;DR I have Claude code hooked up now, I am not sure there is much that an “Official” AI bot could do that I can’t do now.
1 Like
Tell that to these guys OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack
1 Like
And these: Gemini Hacked Three Companies in First Known Breakout by Google’s AI
and more to come…
1 Like
I have say I near wept with joy as the Spook items on this page started arriving. The simplicity it has provided to my system is astounding. Thank you so much Frenk, I know you’re starting by solving your problems, but as you acknowledge in some of the descriptions for these additions - we’re all going to benefit.
You’re a star 
1 Like
Using low end computers unexpectedly places us at the forefront of cybersecurity and privacy protection (HP T620 user here!)
Jokes aside - I don’t think the idea of an official AI in Home Assistant is a bad one. I believe many users would welcome it and subscribe. Provided it’s a local installation (@nabu), rather than just a redirect to one of the existing cloud-based AIs (which just steal all your data).
If I ‘up graded’ to an HP t630 or similar, what the heck do I do with the 2gb Rpi 4?
To run any local AI you will need a competent video card. For that, think competent gaming PC or equivalent. The GPU is the most important piece in this case.
I build my system to be responsive on a 16GB GPU. What you ACTUALLY need depends on a billion things including how much context your setup needs. Any more is out of scope for this particular discussion.
As. Far as should they.
ABSOLUTELY NOT.
I think they’re doing exactly the right thing right now focusing on the plumbing to make it easier for everyone to build what they prefer.
I do NOT think they should be in the business of providing LLMs but they should provide the hooks and guidance for people to get a basic experience and expand. (like assist and byo integration)
They cannot help but make the same mistakes a google or Amazon will because they have to optimize for what someone else wants or the safe route - (my thoughts on why Alexa sucks are very relevant here.) Friday's Party: Creating a Private, Agentic AI using Voice Assistant tools - #788 by NathanCu
Because I STILL firmly believe (two years in this ride now) that We STILL haven’t landed on the right shape. It still takes too much context the way HA does tool loading and there isn’t enough security for me personally inthe base tools… Too much needs to be exposed for basic functionality and each exposure reduces security and raises default context size. That has to change…
Yet to trust it to a general purpose base AI assistant you can trust for every situation (which is where it needs to be before Nabu can bet the business on it if they decide to provide the base agent… And Yes it’s a bet and yes if they provide it they’re liable for what it does etc…)
All that complexity and international law to me screams provide good plumbing but the user must do it themselves else liability central. Anything else is too dangerous to the business.
1 Like
@johnBoy Sure, you can keep the 2 GB Pi 4.
But you probably wouldn’t want to run the actual LLM on it, apart from some very small models. But there’s no reason the model has to run on the HA box in the first place.
It could just as well run on another machine on your LAN with Ollama/llama.cpp, or use something remote like OpenAI, Claude, etc.
The Pi would basically just keep running HA as usual and talk to the LLM somewhere else.
I wasn’t thinking of using AI at all.
I’m just thinking this Pi is going to run out of steam shortly. With all the extra 'stuff ’ that gets added - which is good - to HA. Unless some things are optional installs.
I would guess, things like video use a lot of memory and cpu use? Just talking to AI might?
But I don’t actually know how these things work within an HA system.