Managed AI/LLM in Nabu Casa Cloud for Assist?

Configuration Voice Assistant
, , , ,
1

For me personally, this is currently the biggest missing piece of Home Assistant Cloud.

I would really like to have a managed AI/LLM service for Assist part of the HA Cloud subscription.

I already host essentially everything myself, including remote access and the rest of my Home Assistant infrastructure. Because of that, the current Cloud subscription doesn’t provide enough value for me to justify the cost just for STT/TTS.

However, I would actually be very interested in paying for Cloud if it provided a fully managed AI voice experience.

What I currently do

At the moment I use third-party/custom integrations with Microsoft for STT/TTS and Gemini using its free tier for the conversation agent.

This works reasonably well, but it has exactly the problem I would like a Cloud subscription to solve:

  • APIs and models change
  • free-tier limits can change
  • models get deprecated or replaced
  • integrations occasionally need adjustments
  • I have to keep track of which model currently works best
  • I have to maintain the whole setup myself

Because my current hardware is by far not good enough to host an LLM by myself, I need to use external cloud services. And I don’t want to spend time managing those, they should just work (which they often don’t).

What I would like from Nabu Casa

I’d like Nabu Casa taking care of the underlying models and infrastructure.

I wouldn’t even necessarily care which model is being used. In fact, I would prefer not having to care or having clear recommendations depending on what I want.

If Nabu Casa switches models, improves the model, changes providers, or replaces an API, that should simply happen transparently without requiring me to change my configuration.

I also don’t necessarily expect unlimited usage. A reasonable monthly usage limit would be completely understandable given the variable cost of LLM inference.
But what I would not want is an extra AI tier, because as said, I don’t even use all the other features of Nabu Casa. Either everything must be optional or everything should be included.

My question

Is something like this being considered or discussed internally at Nabu Casa?

I know the current philosophy is to keep LLMs optional and allow users to choose providers such as OpenAI, Google, Anthropic, Ollama, etc. I actually think that is a good approach for a lot of users.

But I think there is also a fairly large group of users who would prefer the opposite:
“I don’t care which LLM you use. I just want Home Assistant Assist to have a good, reliable AI backend that is maintained by Nabu Casa.”

For those users, a managed AI service could make Home Assistant Cloud substantially more valuable.

I’m mainly interested in whether this is something that Nabu Casa could see as part of the future of Home Assistant Cloud, or whether there is a deliberate reason why LLM inference is expected to remain outside the subscription permanently.

I’d especially appreciate an answer from someone involved with Nabu Casa/Assist/Voice, if possible.

2

This sounds like a feature request which will only get attention on the Github Feature Request Discussions

3

Not even there. There’s no planet where they should attempt this. (if you wasn’t them to stay in business) It takes an incredible amount of resources to make this happen that it’s an entire business unto itself the GPUs and infra services iron notwithstanding… Ask OpenAI.

Its a very (and I mean very with an exponent on it) risk laden business and is also currently in turmoil from a pricing perspective right now. (look up the industry buzzword ‘tokenomics’ Unless you can afford to (or because you’re already in it and literally must make your money back) this isn’t a segment I can expect anyone to enter as a service provider unless that van do it at SCALE. Scale here means Google, X, MSFT, AWS hyperscale level…

I can’t in my wildest imagination expect NabuCasa to take this on and for the sake of them staying in business I hope they never.

Don’t hold your breath.

4

I’ve never said they should invest in their own ai infrastructure lol.
What I meant was that they could provide an AI in a easy to use way and included in the sub, not hosting the infrastructure behind it.

5

I understand what you said and it’s not that easy. That would have to be a pass through service. Again incredibly risky, especially with service commits and pricing. (all hyperscale providers that do this ask for a commit of volume to enter said deal.)

I understand what you might LIKE. BUT, what your asking is incredibly impractical without a huge influx of cash. (yes even as a passfhrough provider, see previous about service commit levels.)

Reality. Let’s ignore for a moment that the OHF is actively trying to get folks to move local. You’d have to get a service provider to provide discounted service to Nabu clients to make it economically feasible (even for NC to do it at a net breakeven, things cost money to run…)

Ill be very clear here… They (hyperscalers) will only do that of they get something out of the deal. (it would likely mean a deal for your info - which is against the charter of the OHF.)

At which point Nabu would have no way to provide the service without risking the very reason they exist.

Again. Don’t hold your breath.

There are plenty of reputable providers that are available already for OHF to go there. It’s much better for them to point and say go there, we enable you.

Personally (and this is my day job) I’d rather have a DIRECT contract with a reputable provider that puts in writing what they will and will not do with your data. And if your curious about that see what I can do with your data in ZenOS or Friday’s Party.

6

Honestly I do not understand (other than $$$) why they promote the cloud so heavily anyway. Home Assistant is supposed to be about LOCAL. I’ve spent weeks (as have many) working to get myself free of Google cloud. Why migrate that to someone else’s cloud?

I justify the cloud primarily to support OHF/HA/Nabu Casa or where ever the money goes.

7

Not an answer on whether Nabu Casa should do this, but a data point on the cost worry.

The device-control part of Assist does not need a generative model. Priced on its own it is small: I measured about $0.00006 a command for on, off, toggle, brightness and state questions, with everything else handed to a fallback agent. So the monthly figure people are afraid of here is mostly the conversation, not the commands.

8

I think it’s a little misdirected to say “afraid of”.

Many of us resist the idea of cloud based home control, not the cost of it.

What I commented on and feel is a bit disingenuous of OHF is them promoting THEIR cloud while simultaneously making a mantra of “local”.

9

OHF mantra, if it is that, is " privacy, choice, and sustainability". Local control and local only access should be a choice, something that Google Cloud will never offer you. Nabu Casa Cloud is a choice, as is using each of the services they offer.

I really don’t think that comparing the 2 is fair.

10

Your point is fair, but I think it’s also fair to say that “cloud is cloud” – any cloud means:

  1. It won’t work when the internet is down
  2. Someone else will have a lot of details about you and your data.
  3. Anyone can be hacked and probably will be. See (2).

Now, Nabu Casa is not trying to suck every possible bit of information from you and turn you into the product, they aren’t making a mockery of “don’t be evil”, and even though I disagree with their priorities in many ways, I think they are VASTLY different from Google/Amazon/etc.

But… a cloud is still a cloud.

11

No doubt. Pick your cloud wisely if you need one, if you don’t then don’t pick a cloud.

Personally I need access to my front door camera feeds, so cloud it is and Nabu Casa makes my brain hurt less trying to get that feed… stun and turn are beyond my skill set.

12

Indeed. I also have several devices I can’t manage without a cloud, for example my LG Appliances (there are local alternatives they just don’t work for my devices), or my Anker whole house battery. But it’s good to fight the good fight where you can.