The current state of AI support in HASS is more than a little frustrating. The new ‘LiteLLM’ integration kind of pushed me over the edge on this issue so please forgive me if I am shitty. I love HASS and I want it to be better.
There are currently 5.2 trillion AI integrations spread across the official sources and HACS. This is an estimate, but close enough.
Some add-ons hard code everything. Fields are missing, some fields are set to required when they shouldn’t be. Integrations duplicate each other or fail to recognize the simplicity of consolidation.
The LiteLLM integration (seems to) ONLY support conversation agents…Why??? It’s an LLM provider aggregator and supports…well everything. Here’s the thing…We do not need a LiteLLM integration. Nor do we need one for Openrouter. Nor do we need one for OpenAI TTS. Nor do we need one for whatever.
[One] OpenAI API compatible integration can handle all of this. LiteLLM, OpenAI, Openrouter and other endpoints as well. It can provide conversation agents, data generation, image generation, STT, TTS, etc etc etc.
Like, Gemini and Anthropic and Ollama (although Ollama also supports the OpenAI API standard as well) all have their own BS. Fine. Those are good. Leave them, but are they also providing ALL the services they can? Well, the Gemini integration does and bless’em for it.
So can there be a plan or effort to clean up the space, at least a little? Why does the integration “OpenAI GPT-4o Mini TTS” even exist at this point?
IDK. I’m ranting at this point. At least you know AI didn’t write this spaghetti.
And tbh I am not exactly sure what the rant is about.
There are multiple integrations because there’s multiple services, multiple ways to connect and honestly not everyone does it the same. Want grok great. I personally don’t care for it but if you want it I want you to be able to have it.
If someone needed a litellm integration (which I was actually happy to see it I’m experimenting with it now and it does solve a specific issue… ) they can use it. I’m sure it only supports conversation agents because it’s literally brand new.I don’t see any problem with driving choice.
In fact if I had my way I’d prefer a conversation agent that allows me to turn off tools in the assist pipe and allow me to edit the system prompt. For that I need a custom from HACS (Skye’s localLLM does it by basically forking ASSIST itself)
Its a custom integration it’ll probably be there forever. Someone wrote it because it solved a specific issue. Whether it exists as an issue anymore… Who knows probably not that was originally for enabling a streaming OAI voice provider. But I haven’t looked to see if it’s updated recently. They may have added 87 new features by now.
HA ITSELF Does not control HACS (Home Assistant Community Store) so as long as someone in the community cares to support the git repo that it uses. It’ll probably stay.
So what exactly does ‘clean up the space’ mean? I’m sure those with the powers that be would listen to a good plan?
There’s plenty of issues with AI, and LLM in general in HA. But I’ll never complain about choice.
I am saying, essentially, we need fewer, more generalized integrations that support all the features. One OpenAI API compatible integration that’s configurable could eliminate 4, maybe 5, current integrations. Hard coding integrations that solve one need has created a mess and makes the integration of AI into Home Assistant WAY less user friendly. That’s what HASS is supposed to be moving towards isn’t it? Cleaner, more user friendly for everyone. They way I have had to integrate the services I have is hacky at best.
Also, I fully understand that HACS lies outside the HASS administered envelope. My point in mentioning it is that one-off solutions don’t just bleed into the official integrations but also into HACS as people try to solve problems and enhance compatibility that’s all due to the fragmentation I mention.
You can’t solve everything with one universal integration sorry. It simply doesn’t work that way
The way ollama, docker model runner, vllm, all have unique quirks. Oai and Anthropic are always introducing the next cool thing.
The needs of a multi model platform or a router like litellm have wholly different needs.
You need SME thay understand each integration for the integration to be the best they can be… That WILL NOT happen with one jack of all trades (master of none) integration.
If there’s one. Who makes the decision of what that one needs. Sorry. As a practitioner on the space I wholeheartedly disagree.
Then of course there’s no single developer people write what they like and indeed those devs offer up thier stuff to core of they want to. Yes core paid dev does drive some directions but as a whole once the pattern is set it’s up to the volunteer devdlopers to bring the features… It’s not in the community interest to stop someone from offering a new Integration and it’s also foolish to think one integration can ever do it all.