I sort of waffled on if this is a HA question or a MA question, but I think it’s an HA question.
I’ve been trying to get the LLM integration working between voice assistants in HA and a Music Assistant install, and have run into some issues. I have MA’s HACS integration tied to a ChatGPT conversation agent, and it seems to work perfectly fine. I can send pretty complex requests to it, and it does the right thing the majority of the time.
The core isuue I’ve got is that I run Google AI as the LLM in my HA install, because the HA ChatGPT integration is so wordy and uses so many tokens, making ChatGPT especially expensive. Particularly as compared to “free”, which Google AI is. The MA integration basically doesn’t work with Google AI – it only sporadically understands and the response doesn’t seem to trigger the intent and actually cause something to happen, You just get JSON text as a response. So, I’m kinda stuck with ChatGPT for MA and Google AI for HA.
The problem I seem to have is that voice commands sent to Google AI seem to only occasionally cause the MA integration to happen. So if was to say, as an example “Play the first two songs from Pink Floyd’s second album on the kitchen speakers”, ChatGPT on HA will trigger the right command to happen in MA and it works fine. That exact same command sent to Google AI results in “I am sorry, I cannot fulfill this request. I cannot search for music.”
The docs aren’t especially clear how the execution path works jumping between the voice assistant config, the conversation engine, how that works with regard to intents and triggering things in integrations like the Music Assistant one. Like there’s clearly something being told to the LLM that ChatGPT is understanding and Google AI isn’t.
Does anyone know if there’s any way to get this to work? (I suspect the same issue would be true with someone using an local LLM, as well?)