[Voice Assist] No final Message Problem

Configuration Voice Assistant
1

Hi, I’m using the Assist pipeline in Home Assistant with an LLM conversation agent.

When I ask it to perform an action, such as turning on a light, the tool call succeeds. Sometimes, however, the agent does not send a final text response like “The light is now on.” I’m fine with that and would prefer Assist to remain silent in this case.

The problem is that the pipeline treats the tool call as an incomplete response and outputs “Unable to get response”, because the last response was a tool call rather than a text message.

Expected behavior: If the tool call completed successfully and there is no final text response, the pipeline should treat this as a successful interaction and simply skip TTS.

Actual behavior: The action succeeds, but Assist reports “Unable to get response.”

Is there a way to make Home Assistant accept a successful tool call without requiring a final text response?

2

Check and see what it responded.

If the tool succeeds and doe some reason the assist engine doesn’t understand what came back. Same effect as a timeout. Which means…

^^^ that error msg.

I don’t know the e, act issue from your description but im 100% positive whatever it is appears like a timeout to the HA conversation pipe. So check out the specific tool call.

(I had one a couple of months ago and the response looked ok but was extremely minorly malformed (I think it was a misplaced comma or quote inthe tool resp it was. Months ago don’t remember the exact issue but if I do I’ll post it.) and of you ran it a certain way… This…

It took me yet another month to track it down.. Malformed respi. Fortunately it was my tool so once I found it, I was able to fix it.

Youll have to fine tooth comb the voice response log comparing good calls to bad.

TL;Dr tool call that was doing this to me was indeed faulty just so minor it took another month to find. This is what HA does when a tool call that should produce voice response times out or appears to.

3

As I said, the problem isn’t a “wrong” answer or anything like that, but simply the way some AI models respond.

Older models, in particular, would execute commands (Call intent__HassTurnOn) and then respond with something like “The light is now on” or “The garage door is now open” (or, in the example below, simply “Done”).
Most newer models, however, sometimes respond differently. Yes, sometimes they, too, provide a text response at the end, but sometimes they execute their tools and then stop. Home Assistant can’t handle this, though, because it expects a text response (see first image). No text response = “Unable to get response,” which is nonsense, since the tool/task was completed beforehand, the AI just didn’t “think” it necessary to confirm it again (which makes sense; I can see when the light turns on, so I don’t need to get that as a response).

So the problem isn’t a faulty response, but simply that Home Assistant can’t handle it when no text is returned in the Assistant pipeline.

image
image614×388 13.1 KB

image
image620×444 14 KB

4

I can’t reproduce that. Have you opened an issue against the conversation integration in core?

5

No, not yet, I’ll do that when I have time.
What exactly do you mean by “you can’t reproduce it”? What happens for you when the AI agent responds using only tools and no text?

And FYI, this screenshot is from a self-hosted AI web UI that, like Home Assistant, uses the OpenAI API. Here, too, it often happens that the AI simply executes tools. That’s normal behavior for the API :slight_smile:

image
image305×148 5.9 KB

6

well, i found this:

Someone has already reported this issue.

8

What I mean is even if I TRY to force it mine doesn’t error. So exactly - I cannot reproduce the issue you are seeing. It means exactly what it says. The only time I see it is when a tool blows up and errors and that error makes it to the voice pipe.

Follow that report and add info of you have any relevant to add.