In the recent 2024.4 update, it was announced that HA scripts can now be exposed to the Assist API and consequently to LLMs.
From my understanding, script access was presented as “tools” or a way to guide the LLMs in finding the best action to perform for a given intent. More details in the official blog post.
Do LLMs have access to the script results using response_variable after invoking them?
I have the same question. I have been trying to get the Ollama agent to give a weather forecast based on a custom script response, but it never seems to give a correct answer.
The script itself is called correctly with the correct parameters and also returns the correct response. But the conversation agent either responds with made up forecasts, or with a message that it cannot answer the question.
So I found a way, but not by having the LLM call the script directly, but through an intent script which is exposed to the LLM.
The intent script returns the scripts response in the speech: text.
intent_script:
YourIntentScript:
description: "Intent Script Description"
action:
- service: script.yourscript
data:
parameter: "{{ parameter }}"
response_variable: result
- stop: ""
response_variable: result
speech:
text: |
{% if action_response %}
{{ action_response }}
{% else %}
Could not get a response
{% endif %}
It seems the text does not necessarily have to be a text. In my setting, the LLM picked up a dict returned from the script just fine.
fiy I noticed using {{ action_response|tojson }} helps to get better answers from the LLM
Thanks! I’ll give that a try! 
Has this ever been resolved?
My LLM doesn’t seem to be able to see the results from the response_variable it says it’s empty or an empty object, while the log output says otherwise.
I tried working with the intent, but to my knowledge I cannot add a ‘custom’ variable to it… ?
i am also looking for a solution or a simple example of returning the response of an intent back to speech → text
what would a hello world example script look like?
I tried various ways to communicate with the LLM, the only succes I got was with a response_variable. I’m still in the dark about any other ways to be able to address the correct conversation.
You can see my repsonse_variable example here:
Btw I had more luck with exposing a script to the assistant, and explaining the syntax in the prompt than I did with intent. I also have no idea if intent is actually functional? (When I ask my LLM what I can say, it has no idea).