I’ve got the Voice PE setup and working. I can turn on and off a few specific lights I’ve exposed and even use voice commands to activate a few pyscript scripts I’ve setup and return voice responses.
Now I’ve installed the “Extended OpenAI Converstation” integration via HACS in preperation for trying what came out of this thread. However, so far I’ve had limited success in sending anything to it.
I’ve got it setup (named “ChatGPT”) under “Settings > Voice Assistants”, and if I click on “Start Conversation” from it’s context menu, then it seems to work.
However, the way it’s setup, I thought that if the local assistant didn’t understand a request, then it would send it off to an external one, if available. (I have the “Prefer handling commands locally” set for the ChatGPT assistant.) It doesn’t seem to be doing this. If I ask the Voice PE something like “What’s two and a half pounds in ounces”, it just responds, “Sorry, I’m not aware of any devices called two and a half pounds in ounces”.
However #2, that’s not even what I really want, but I’m not sure if what I really want is possible. I don’t actually want it to automatically fall back to some external LLM if it can’t figure out what I’m talking about. In that case I’d prefer it do what’s it’s doing. What I really want is explicit control over what goes to which voice assistant. I want to be able to say, “Ok, Nabu, ask ChatGPT what’s two and a half pounds in ounces?” And even further, the Extended OpenAI Conversation addon already has the ability to configure multiple “Services” with different intro prompts. So, maybe I’d have one called “Kitchen Assistant” that’s explicitly prompted to answer kitchen related questions, and another called “Spanish Homework Assistant” that’s prompted to help with questions related to learning Spanish. Then I could say something like “Ok, Nabu, ask the Spanish Language Assistant what’s the difference between Que and Cuál?”
Any thoughts for ideas? Have I just overlooked some configuration setting somewhere that makes this all possible? Thanks!
Problem #1: The Voice PE device itself is configured in HA to use a specific assistant, and that’s all it’s going to use. So, once I set that to the ChatGPT assistant, itself configured to first try the local assistant, then that part worked as I expected. It tries the local assistant if that setting is enabled for the external assistant, then goes to the external one. So, that was me just not understanding how this was supposed to work.
Problem #2: As I first said, I really want more control over which assistant commands go to, and I think I’ve figured that out. From an automation, you can set/change which assistant the device is using.
So, I have the Voice PE device set to use the local assistant, then I have an automation set to trigger on the V-PE hearing “Turn on ChatGPT”, this then updates the device to change it’s assistant to the ChatGPT one. Then it also set the response to “ChatGPT is listening”
Now, I can say “Ok Nabu, How many pounds are in an ounce”, and ChatGPT answers the question.
Pretty sure I know the path from here to create other assistants with their own specialties that I can activate with commands like “Ok Nabu, turn on Spanish Homework Assistant”.
The thing i’m not sure of now is how to set some sort of timeout, so that after a certain amount of time of no activity, it automatically switches back to the preferred assistant. Currently I’ve got an automation listening for “Turn off ChatGPT” that does so, but there’s no way I’m going to remember to do that every time.
The thing i’m not sure of now is how to set some sort of timeout, so that after a certain amount of time of no activity, it automatically switches back to the preferred assistant. Currently I’ve got an automation listening for “Turn off ChatGPT” that does so, but there’s no way I’m going to remember to do that every time.
Well, never mind, I figured that out, too. Can set an automation to watch for the Voice Media Player to go idle for 15 seconds, and then if the device is set to use anything other than the preferred asstant, to switch it back to the preferred one and send an announcement saying so.
