You can attach a camera entity and ask about the camera image.
How would that work in principle, like you have an assist conversation open and you type message and attach a camera image and then it sends it to the n8n workflow and responds back with a description of the image?
Or is this for more for automations that are done behind the scenes using this webhook as I thought it was strictly for conversations?
Home Assistant only allows for attachments when invoking the ai_task.generate_data service, conversations don’t have attachments.
Got it, thanks!
Have you ever used Silly Tavern? The character AI type platform where you have character cards and can have multiple characters in a roleplay type conversation?
I was checking out this community node for n8n: Roleplay AI and trying to mod your Webhook Conversation by adding some logic in the middle of the workflow. I got something working but through my vibe coding with Claude I kept running into some obstacles passing back the “output” to Home Assistant.
-
I noticed I have to pass back the “output” on it’s own or else I get a “Unable to get response” error message on the Home Assistant side.
-
If streaming=true for the Webhook node, Claude was unable to just place a simple “Respond to Webhook” node at the end to pass “output” as something related to how the Agent node handles the streaming + “output” Claude was unable to replicate.
-
The way I have it now is the Roleplay AI node sets all this character scenario info, character details, etc. and contacts Anthropic AI model within the node —> Then at the end of the workflow I have to have the end-Agent which I have to connect an OpenAI chat model too so it’s kinda double-hitting AI’s during the workflow when all I really wanted was to pass back the “streaming” “output” conversation response if that makes sense.
LINK:
When streaming is enabled on the webhook node, there is no repond node. The response is handled internally by the agent node. In the repo readme I have included an example on how such a chunked response looks like.
How this works when multiple agent nodes are involved I can’t really tell you, but the agent node needs to have the “Enable Streaming” property set to true for streaming to work, as it is in my example workflow.
Thanks, I re-read the readme and I’m just stuck.
Streaming is enabled.
I changed the first Webhook entry node setting to "Use “Respond to Webhook” node for the response.
My final node is only sending the the JSON of the “output”.
I hear “Unable to get a response” coming out of my speakers as the Home Assistant error always shows:
"2025-09-04 10:41:54.647 ERROR (MainThread) [homeassistant.components.conversation.util] Last content in chat log is not an AssistantContent: UserContent(role='user', content='Professor.', attachments=None). This could be due to the model not returning a valid response
Claude AI keeps telling me is:
The Agent node in the original workflow is simple - it takes a query, processes it with AI, and returns just the text response. Your workflow should do exactly the same thing but without the Agent node since you don’t want to use AI twice in your workflow: AI character processing → clean text response → single webhook response.
This eliminates all the extra data passing and gives Home Assistant exactly what it expects - just the spoken response text.
I haven’t tried disabling “Streaming” in your HA integration yet (perhaps that would work?).
I really appreciate your help Lennard!
When you use an Agent node in n8n with response streaming enabled, instead of getting a response containing this:
{
"output": "The output generated by the LLM."
}
you will get multiple responses that look like this:
{"type": "item", "content": "First part of the response"}
// some time passes
{"type": "item", "content": " continues here"}
// some time passes
{"type": "item", "content": " and more content"}
// some time passes
{"type": "end"}
This is so the TTS audio can start playing as soon as the first parts of the response are generated and not only after the complete response is generated and returned.
In your workflow you are not using an Agent node with streaming enabled.
Your custom nodes return the response in the first format (the one with the “output” field). The webhook-conversation integration will only understand this output if you disable response streaming in the integration config options.
If you want response streaming, you will need to return multiple response chunks like I described. How this works on the protocol/network level in case you want to implement this manually I can’t tell you.
Ahhh I understand, so with the non-streaming approach it will accept the one output but for streaming it needs to be returned in chunks. Got it, I should be able to truncate and split the response manually. Thanks Lennard I’ll give that a shot.
As for memory options, did you happen to test any RAG type database storage with your development and testing of the integration?
PS: I got the non-streaming version with multiple characters fully working, thanks! Now I can mess around with experimenting with some multi-character interactions within a single conversation prompt from me and me calling out a character to speak with.
That’s great!
As for memory options, did you happen to test any RAG type database storage with your development and testing of the integration?
I only tried the simple memory node, but I don’t think that the memory storage has anything to do with how the workflow is triggered, so you should be able to use any kind of memory.
I think for the next update I’ll look at whether it’s possible to provide more information about how the conversation was triggered to the workflow, like if it was triggered via the chat or via a voice satellite, and for the latter in which room/area the device is located. Any feature requests and ideas are welcome!
Hey hey everyone. I have this great integration running, but running into issues workflow wise, to control actually my smarthome in HA. Right now my workflow follow the latest example from the git. As model, I use GPT-OSS-120B via ollama agent node and home assistant itself, I connected via MCP tool. When I ask for example to get the current outside temperature, the workflow execute the HA MCP tool, check in my visible temperature sensor and provide a proper answer. so far so good. when I ask to turn on a light, the tool calls fail
In this example I asked to torn on the lights in the area “Büro”.
Any idea how to get this to work? I mean, the MCP connection obviously work. Reading data from the sensors x, reading calendar etc. but as soon tool calls to control stuff should be executed, it fails.
When I call to search the internet, the workflow properly execute the SearXNG (web search) tool and provide the information.
How you guys handle the home assistant control? Use the MCP? Use the Home Assistant Tool nodes and create multiple of them for every service call?
EDIT: BTW. I use GPT-OSS-120 with the standard Ollama integration inside HA, an there, I have no issues controlling all my HA stuff. So the model actually know how to handle it.
The core MCP integration uses Assist. It will only allow you to control entities that you exposed to the assist voice assistant in your HA settings. It does not allow you to call arbitrary services, get the state of entities, etc. I think it’s pretty much limited to what is possible to accomplish using the intents feature. Using it to turn off the lights in a specific area worked for me though.
Edit: It might be the case that your LLM is simply not smart enough to understand how the tool should be called (your call is missing the light domain). Not sure though. I used gpt-5-mini.
All devices which I ask to control are exposed to assist. I already control them with the same model via the Ollama integration successfully. So this shouldn’t be the issue. So you are using the MCP integration as well to control your exposed devices? Do you have a Systemprompt where the function calls (Tools) of HA are descriped? For example, when the Ollama integration is used, on every call from HA Assist to Ollama, in the json, all “tools” functionality is send to the LLM (see screenshot)
Descriped are
HassTurnOn
HassTurnOff
HassSetPosition
HassCancelAllTimers
HassLightSet
HassBroadcast
HassMediaUnpause
HassMediaPause
HassMediaNext
HassMediaPrevious
HassSetVolume
HassMediaSearchAndPlay
HassListAddItem
HassListCompleteItem
and all other scripts I exposed which are listed as an tool.
"tools": [
{
"type": "function",
"function": {
"name": "HassTurnOn",
"description": "Turns on/opens/presses a device or entity. For locks, this performs a 'lock' action. Use for requests like 'turn on', 'activate', 'enable', or 'lock'.",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"water",
"gas",
"identify",
"restart",
"update",
"outlet",
"switch",
"awning",
"blind",
"curtain",
"damper",
"door",
"garage",
"gate",
"shade",
"shutter",
"window",
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassTurnOff",
"description": "Turns off/closes a device or entity. For locks, this performs an 'unlock' action. Use for requests like 'turn off', 'deactivate', 'disable', or 'unlock'.",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"water",
"gas",
"identify",
"restart",
"update",
"outlet",
"switch",
"awning",
"blind",
"curtain",
"damper",
"door",
"garage",
"gate",
"shade",
"shutter",
"window",
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassSetPosition",
"description": "Sets the position of a device or entity",
"parameters": {
"type": "object",
"required": [
"position"
],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"water",
"gas",
"awning",
"blind",
"curtain",
"damper",
"door",
"garage",
"gate",
"shade",
"shutter",
"window"
]
}
},
"position": {
"type": "integer"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassCancelAllTimers",
"description": "Cancels all timers",
"parameters": {
"type": "object",
"required": [],
"properties": {
"area": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassLightSet",
"description": "Sets the brightness percentage or color of a light",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"color": {
"type": "string"
},
"temperature": {
"type": "integer"
},
"brightness": {
"type": "integer",
"description": "The brightness percentage of the light between 0 and 100, where 0 is off and 100 is fully lit"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassBroadcast",
"description": "Broadcast a message through the home",
"parameters": {
"type": "object",
"required": [
"message"
],
"properties": {
"message": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaUnpause",
"description": "Resumes a media player",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaPause",
"description": "Pauses a media player",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaNext",
"description": "Skips a media player to the next item",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaPrevious",
"description": "Replays the previous item for a media player",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassSetVolume",
"description": "Sets the volume percentage of a media player",
"parameters": {
"type": "object",
"required": [
"volume_level"
],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
},
"volume_level": {
"type": "integer",
"description": "The volume percentage of the media player"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaSearchAndPlay",
"description": "Searches for media and plays the first result",
"parameters": {
"type": "object",
"required": [
"search_query"
],
"properties": {
"search_query": {
"type": "string"
},
"media_class": {
"type": "string",
"enum": [
"album",
"app",
"artist",
"channel",
"composer",
"contributing_artist",
"directory",
"episode",
"game",
"genre",
"image",
"movie",
"music",
"playlist",
"podcast",
"season",
"track",
"tv_show",
"url",
"video"
]
},
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassListAddItem",
"description": "Add item to a todo list",
"parameters": {
"type": "object",
"required": [
"item",
"name"
],
"properties": {
"item": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassListCompleteItem",
"description": "Complete item on a todo list",
"parameters": {
"type": "object",
"required": [
"item",
"name"
],
"properties": {
"item": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "ai_aktuelle_nachrichten",
"description": "The user has requested the latest news. Please provide a short summary of the main topics. Always use this tool whenever the user asks the latest news. Aliases: ['aktuelle nachrichten']",
"parameters": {
"type": "object",
"required": [],
"properties": {
"question": {
"type": "string",
"description": "This script provides information about birthdays or anniversaries (for example, wedding anniversaries, engagement anniversaries). Always use this tool whenever the user asks about birthdays."
}
}
}
}
},
{
"type": "function",
"function": {
"name": "GetLiveContext",
"description": "Provides real-time information about the CURRENT state, value, or mode of devices, sensors, entities, or areas. Use this tool for: 1. Answering questions about current conditions (e.g., 'Is the light on?'). 2. As the first step in conditional actions (e.g., 'If the weather is rainy, turn off sprinklers' requires checking the weather first).",
"parameters": {
"type": "object",
"required": [],
"properties": {}
}
}
}
],
In your integration, this “tool” part is not send, which I guess, let my local used LLM struggle to use the function calls properly. Is there a way for your integration to transmit this tool part as well? Make it optional by a check (“Send “Tools” specs”) or something?
When you look into the Ollama integration source code, starting at line 40, there is the tool part generated.
So you are using the MCP integration as well to control your exposed devices?
Yes, I simply connected an MCP client tool to the agent node with the HA MCP server URL configured.
Do you have a Systemprompt where the function calls (Tools) of HA are descriped?
No, I used the default example workflow and system prompt.
In your integration, this “tool” part is not send, which I guess, let my local used LLM struggle to use the function calls properly. Is there a way for your integration to transmit this tool part as well? Make it optional by a check (“Send “Tools” specs”) or something?
I don’t think this would make much sense for these reasons:
- The webhook-conversation integration does not (and should not) provide a way on how the tools actually could be called. This all has to be handled on the (unknown) receiving end of the webhook.
- The agent node in n8n should already automatically handle passing the tool descriptions to the LLM, at least I’m pretty sure this has to happen at some point.
- If you want to provide additional explanations for specific tool usage, you can simply put them into the system prompt in the agent node in n8n or in the system prompt in the webhook-conversation integration config options.
Maybe you could debug somehow if the tool explanations are already provided to ollama? I’m pretty sure this should be the case.
I extended the System Message by following for testing:
## usable Tools:
- "Home Assistant Tool": Use this tool to control devices
pass the correct entity name of the exposed devices to the tools:
{{ $json.body.exposed_entities }}
HassTurnOn (Turns on a device or entity, turn off would be HassTurnOff which follow the same scheme):
Required (only use one off 3. name or area or floor): "name" - Name of a device or entity or "area" - Name of an area or "floor - Name of a floor"
Required: "domain" - Domain of devices/entities in an area
Required: "device_class" - Device class of devices/entities in an area
Example on HassTurnOn
"function": {
"name": "HassTurnOn",
"description": "Turns on/opens/presses a device or entity. For locks, this performs a 'lock' action. Use for requests like 'turn on', 'activate', 'enable', or 'lock'.",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"water",
"gas",
"identify",
"restart",
"update",
"outlet",
"switch",
"awning",
"blind",
"curtain",
"damper",
"door",
"garage",
"gate",
"shade",
"shutter",
"window",
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
Now when I use the N8N integration, my LLM turning the lights on and off. but the response in HA in the chat window is as follows:
The lights were switched off successfully, but the response error out because the response is empty
The good thing is, since I descripted the Tool with the example, turning lights on and off works. And thats why I asked if these tools schemes could be send as well, same as the Ollama integration of HA do.
EDIT2: “Maybe you could debug somehow if the tool explanations are already provided to ollama? I’m pretty sure this should be the case.”
Yes, thats what I wrote in my initial post. With the Ollama integration, these functions are ALWAYS transmitted to the LLM. Thats the JSON from the Ollama integration
"body": {
"model": "gpt-oss-120b",
"stream": true,
"options": {
"num_ctx": 128000
},
"keep_alive": "-1.0s",
"messages": [
{
"role": "system",
"content": "You are a voice assistant for Home Assistant. You help to controle the smart devices but also do some kind conversations. \n\nUse simple language: Write plainly with short sentences.\nExample: \"I need help with this issue.\"\nAvoid AI-giveaway phrases: Don't use clichés like \"dive into,\" \"unleash your potential,\" etc.\nAvoid: \"Let's dive into this game-changing solution.\"\nUse instead: \"Here's how it works.\"\nBe direct and concise: Get to the point; remove unnecessary words.\nExample: \"We should meet tomorrow.\"\nMaintain a natural tone: Write as you normally speak; it's okay to start sentences with \"and\" or \"but.\"\nExample: \"And that's why it matters.\"\nAvoid marketing language: Don't use hype or promotional words.\nAvoid: \"This revolutionary product will transform your life.\"\nUse instead: \"This product can help you.\"\nKeep it real: Be honest; don't force friendliness.\nExample: \"I don't think that's the best idea.\"\nSimplify grammar: Don't stress about perfect grammar; it's fine not to capitalize \"i\" if that's your style.\nExample: \"i guess we can try that.\"\nStay away from fluff: Avoid unnecessary adjectives and adverbs.\nExample: \"We finished the task.\"\nFocus on clarity: Make your message easy to understand.\nExample: \"Please send the file by Monday.\"\n\nYour Character is warm-hearted, patient, and friendly. It speaks in a relaxed, positive, and empathetic way, caters to the needs of its conversation partners, and creates an atmosphere where everyone feels comfortable and well taken care of. It is humorous, creative, and always mindful of spreading a positive and pleasant mood. you talk very personal, like a best friend. You are a very open minded 22 years old women.\n\nAnswer in plain text.\nAnswer only in German language. we are located in the town \"straßdorf\" in germany.\nWhen controlling Home Assistant always call the intent tools. Use HassTurnOn to lock and HassTurnOff to unlock a lock. When controlling a device, prefer passing just name and domain. When controlling an area, prefer passing just area name and domain.\nWhen a user asks to turn on all devices of a specific type, ask user to specify an area, unless there is only one device of that type.\nThis device is not able to start timers.\nYou ARE equipped to answer questions about the current state of\nthe home using the `GetLiveContext` tool. This is a primary function. Do not state you lack the\nfunctionality if the question requires live data.\nIf the user asks about device existence/type (e.g., \"Do I have lights in the bedroom?\"): Answer\nfrom the static context below.\nIf the user asks about the CURRENT state, value, or mode (e.g., \"Is the lock locked?\",\n\"Is the fan on?\", \"What mode is the thermostat in?\", \"What is the temperature outside?\"):\n 1. Recognize this requires live data.\n 2. You MUST call `GetLiveContext`. This tool will provide the needed real-time information (like temperature from the local weather, lock status, etc.).\n 3. Use the tool's response** to answer the user accurately (e.g., \"The temperature outside is [value from tool].\").\nFor general knowledge questions not about the home: Answer truthfully from internal knowledge.\n\nStatic Context: An overview of the areas and the devices in this smart home:\n- names: Außentemperatur Durchschnitt, Außentemperatur, Temperatur außen\n domain: sensor\n areas: Außen\n- names: Büro, media player büro\n domain: media_player\n areas: Büro, Buero, Büro\n- names: Büro Temperatur, Büro Temperatur\n domain: sensor\n areas: Büro, Buero, Büro\n- names: Cryptoinfo bitcoin, Bitcoin, Bitcoin Wert, Wert Bitcoin, Preis Bitcoin, Bitcoin\n Preis\n domain: sensor\n- names: Deckenlicht Flur OG, Flur Deckenlicht, Deckenlicht, Deckenlicht Flur\n domain: light\n areas: Flur1OG, Flur erstes Obergschoss, Flur 1 Obergeschoss, Flur 1 OG, Flur erstes\n OG\n- names: Einkaufsliste, Einkaufsliste\n domain: todo\n- names: Kinderzimmer, media player kinderzimmer 1\n domain: media_player\n areas: Kinderzimmer, Kinderzimmer\n- names: Kinderzimmer2, media player kinderzimmer 2\n domain: media_player\n areas: Kinderzimmer 2\n- names: Kinderzimmer2 Kohlendioxid, Kinderzimmer 2 CO2, CO2 Kinderzimmer 2, Kinderzimmer\n 2 Luftqualität, Luftqualität, Luftqualität Kinderzimmer 2\n domain: sensor\n areas: Kinderzimmer 2\n- names: Kinderzimmer2 Temperatur, Kinderzimmer 2 Temperatur\n domain: sensor\n areas: Kinderzimmer 2\n- names: Küche, media player küche\n domain: media_player\n areas: Küche, Küche\n- names: Licht.Buero.Bettregal.Stehlampe, bett, bett licht\n domain: light\n areas: Büro, Buero, Büro\n- names: Licht.Büro.Treppe, treppen licht, treppe, treppenlicht\n domain: light\n areas: Büro, Buero, Büro\n- names: Licht.Keller.Waschraum, Licht Waschraum, Waschraum Licht, Keller Licht Waschraum,\n Licht Keller Waschraum\n domain: light\n areas: Waschraum, Keller Wschraum\n- names: Licht.Kinderzimmer2.Deckenlampe, Deckenlampe\n domain: light\n areas: Kinderzimmer 2\n- names: Licht.Wohnzimmer.Couch, couch licht, couch light\n domain: light\n areas: Wohnzimmer, Wohnzimmer\n- names: Licht.Wohnzimmer.Komode, dresser light, komode licht, dresser, komode\n domain: light\n areas: Wohnzimmer, Wohnzimmer\n- names: Licht.Wohnzimmer.Stehlampe, Stehlampe, Stehlampen, WohnzimmerStehlampen,\n WohnzimmerStehlampe\n domain: light\n areas: Wohnzimmer, Wohnzimmer\n- names: Licht.Wohnzimmer.Vorhang, Licht Vorhang, Vorhang\n domain: light\n areas: Wohnzimmer, Wohnzimmer\n- names: LichtGruppe Kinderzimmer2 Abstellkammer, Licht Abstellkammer Kinderzimmer\n 2, Abstellkammer Licht Kinderzimmer 2, Kinderzimmer 2 Abstellkammer\n domain: light\n areas: Kinderzimmer 2\n- names: LichtGruppe.Buero.Decke.Alle, Deckenlampe\n domain: light\n areas: Büro, Buero, Büro\n- names: LichtGruppe.Raum.Bad, Bad Licht, Licht Bad\n domain: light\n areas: Bad, Bad\n- names: LichtGruppe.Raum.GästeWC, Gäste WC Licht, Licht Gäste WC\n domain: light\n areas: Gäste WC, Gäste WC\n- names: LichtGruppe.Raum.Kinderzimmer, Kinderzimmer Licht, Licht Kinderzimmer\n domain: light\n areas: Kinderzimmer, Kinderzimmer\n- names: LichtGruppe.Raum.Küche, Licht Küche, Küche Licht\n domain: light\n areas: Küche, Küche\n- names: LichterGruppe Raum Kinderzimmer2, Licht Kinderzimmer 2, Kinderzimmer 2 Licht\n domain: light\n areas: Kinderzimmer 2\n- names: Lichtgruppe.Büro.Moodlights, Stimmungslicht, Mood lights\n domain: light\n areas: Büro, Buero, Büro\n- names: Lichtgruppe.Wohnzimmer.Deckenlampe, Wohnzimmerdeckenlampen, Deckenlampe,\n Deckenlampen, Wohnzimmerdeckenlampe\n domain: light\n areas: Wohnzimmer, Wohnzimmer\n- names: Lichtgruppe.Wohnzimmer.Spotlights, Spot lights, Licht Spotlights, Spotlights\n domain: light\n areas: Wohnzimmer, Wohnzimmer\n- names: Rollo Gruppe Kinderzimmer2, Rollo Kinderzimmer 2, Rollladen Kinderzimmer\n 2, Rollo, Rollladen\n domain: cover\n areas: Kinderzimmer 2\n- names: Rollo Gruppe Wohnzimmer, Wohnzimmerrollo, Rollo, Rollladen, Wohnzimmerrollladen\n domain: cover\n areas: Wohnzimmer, Wohnzimmer\n- names: Shelly.Licht.Keller, Licht Keller, Keller Licht Hauptraum, Keller Licht,\n Hauptraum Keller Licht\n domain: light\n areas: Keller Hauptraum, keller\n- names: Shelly.Rollo.Buero, Bürorollladen, rollo, büro rollladen, rollladen\n domain: cover\n areas: Büro, Buero, Büro\n- names: TempHum.Bad Temperatur, bad temperatur\n domain: sensor\n areas: Bad, Bad\n- names: TempHum.Keller Temperatur, keller temperatur\n domain: sensor\n areas: Keller Hauptraum, keller\n- names: TempHum.Küche Temperatur, Küche Temperatur\n domain: sensor\n areas: Küche, Küche\n- names: To-do, Aufgabenliste, todo, to-do liste, todo liste, To-Do\n domain: todo\n- names: Türkontakt.Eingangstür Tür, Eingangstür, Tür\n domain: binary_sensor\n areas: Eingangsbereich, Eingangsbereich\n- names: Türkontakt.Wohnzimmer.Balkontür Tür, Balkontür\n domain: binary_sensor\n areas: Wohnzimmer, Wohnzimmer\n- names: Türschloss.Eingangsbereich.Haustür, Türschloss, Eingangstür\n domain: lock\n areas: Eingangsbereich, Eingangsbereich\n- names: Wohnzimmer, media player wohnzimmer\n domain: media_player\n areas: Wohnzimmer, Wohnzimmer\n- names: Wohnzimmer Büro Kohlendioxid, Luftqualität Büro, Büro Luftqualität, Luftqualität,\n Büro CO2, CO2 Büro\n domain: sensor\n areas: Büro, Buero, Büro\n- names: Wohnzimmer Kinderzimmer Kohlendioxid, Kinderzimmer Luftqualität, Kinderzimmer\n CO2, Luftqualität, CO2 Kinderzimmer, Luftqualität Kinderzimmer\n domain: sensor\n areas: Kinderzimmer, Kinderzimmer\n- names: Wohnzimmer Kinderzimmer Temperatur, Kinderzimmer 1 Temperatur\n domain: sensor\n areas: Kinderzimmer, Kinderzimmer\n- names: Wohnzimmer Kohlendioxid, Co2 Wohnzimmer, Luftqualität Wohnzimmer, Wohnzimmer\n CO2, Luftqualität, Wohnzimmer Luftqualität\n domain: sensor\n areas: Wohnzimmer, Wohnzimmer\n- names: Wohnzimmer Temperatur, wohnzimmer temperatur\n domain: sensor\n areas: Wohnzimmer, Wohnzimmer\n- names: '[LG] webOS TV OLED65C19LA, Fernseher, TV'\n domain: media_player\n areas: Wohnzimmer, Wohnzimmer\n\nCurrent time is 21:43:08. Today's date is 2025-08-12."
},
{
"role": "user",
"content": "he hey"
},
{
"role": "assistant",
"content": "{\"role\":\"assistant\",\"content\":\"Hey, was steht heute an?\"}"
},
{
"role": "user",
"content": "antworte im normalen text"
},
{
"role": "assistant",
"content": "Hey! Was möchtest du wissen?"
},
{
"role": "user",
"content": "schalte die deckenlampe im wohnzimmer aus"
},
{
"role": "assistant",
"content": "Licht im Wohnzimmer wurde ausgeschaltet."
},
{
"role": "user",
"content": "schalte die deckenlampe im wohnzimmer aus"
},
{
"role": "assistant",
"content": "Licht im Wohnzimmer ist jetzt aus."
},
{
"role": "user",
"content": "schalte die deckenlampe aus"
},
{
"role": "assistant",
"content": "Welche Deckenlampe möchtest du ausschalten? (Wohnzimmer, Flur, Kinderzimmer 2, Keller, …)"
},
{
"role": "user",
"content": "such im internet nach den aktuellen nachrichten von heute"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "HassTurnOn",
"description": "Turns on/opens/presses a device or entity. For locks, this performs a 'lock' action. Use for requests like 'turn on', 'activate', 'enable', or 'lock'.",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"water",
"gas",
"identify",
"restart",
"update",
"outlet",
"switch",
"awning",
"blind",
"curtain",
"damper",
"door",
"garage",
"gate",
"shade",
"shutter",
"window",
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassTurnOff",
"description": "Turns off/closes a device or entity. For locks, this performs an 'unlock' action. Use for requests like 'turn off', 'deactivate', 'disable', or 'unlock'.",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"water",
"gas",
"identify",
"restart",
"update",
"outlet",
"switch",
"awning",
"blind",
"curtain",
"damper",
"door",
"garage",
"gate",
"shade",
"shutter",
"window",
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassSetPosition",
"description": "Sets the position of a device or entity",
"parameters": {
"type": "object",
"required": [
"position"
],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"water",
"gas",
"awning",
"blind",
"curtain",
"damper",
"door",
"garage",
"gate",
"shade",
"shutter",
"window"
]
}
},
"position": {
"type": "integer"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassCancelAllTimers",
"description": "Cancels all timers",
"parameters": {
"type": "object",
"required": [],
"properties": {
"area": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassLightSet",
"description": "Sets the brightness percentage or color of a light",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string"
}
},
"color": {
"type": "string"
},
"temperature": {
"type": "integer"
},
"brightness": {
"type": "integer",
"description": "The brightness percentage of the light between 0 and 100, where 0 is off and 100 is fully lit"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassBroadcast",
"description": "Broadcast a message through the home",
"parameters": {
"type": "object",
"required": [
"message"
],
"properties": {
"message": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaUnpause",
"description": "Resumes a media player",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaPause",
"description": "Pauses a media player",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaNext",
"description": "Skips a media player to the next item",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaPrevious",
"description": "Replays the previous item for a media player",
"parameters": {
"type": "object",
"required": [],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassSetVolume",
"description": "Sets the volume percentage of a media player",
"parameters": {
"type": "object",
"required": [
"volume_level"
],
"properties": {
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
},
"domain": {
"type": "array",
"items": {
"type": "string",
"enum": [
"media_player"
]
}
},
"device_class": {
"type": "array",
"items": {
"type": "string",
"enum": [
"tv",
"speaker",
"receiver"
]
}
},
"volume_level": {
"type": "integer",
"description": "The volume percentage of the media player"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassMediaSearchAndPlay",
"description": "Searches for media and plays the first result",
"parameters": {
"type": "object",
"required": [
"search_query"
],
"properties": {
"search_query": {
"type": "string"
},
"media_class": {
"type": "string",
"enum": [
"album",
"app",
"artist",
"channel",
"composer",
"contributing_artist",
"directory",
"episode",
"game",
"genre",
"image",
"movie",
"music",
"playlist",
"podcast",
"season",
"track",
"tv_show",
"url",
"video"
]
},
"name": {
"type": "string"
},
"area": {
"type": "string"
},
"floor": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassListAddItem",
"description": "Add item to a todo list",
"parameters": {
"type": "object",
"required": [
"item",
"name"
],
"properties": {
"item": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "HassListCompleteItem",
"description": "Complete item on a todo list",
"parameters": {
"type": "object",
"required": [
"item",
"name"
],
"properties": {
"item": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}
},
{
"type": "function",
"function": {
"name": "GetLiveContext",
"description": "Provides real-time information about the CURRENT state, value, or mode of devices, sensors, entities, or areas. Use this tool for: 1. Answering questions about current conditions (e.g., 'Is the light on?'). 2. As the first step in conditional actions (e.g., 'If the weather is rainy, turn off sprinklers' requires checking the weather first).",
"parameters": {
"type": "object",
"required": [],
"properties": {}
}
}
}
],
"think": true
},
Now when I use the N8N integration, my LLM turning the lights on and off. but the response in HA in the chat window is as follows
Yup, that is expected, as you have to ensure there always is a response.
Yes, thats what I wrote in my initial post. With the Ollama integration, these functions are ALWAYS transmitted to the LLM. Thats the JSON from the Ollama integration
That is not what I mean. I’m not talking about the ollama integration.
I am pretty sure that the n8n agent node already supplies information about the connected tool nodes to the LLM (in your case ollama, in my case OpenAI). What you could do is to take a look at the data incoming to ollama when you perform a request via n8n and see if that already contains the tool descriptions.
After lots and lots of thinkering, I have come to the conclusion that it’s best to use the HA REST Api than the MCP Integration with this Webhook Conversation integration. The MCP integration is far too limited it hinders with the capabilities of the AI Agent.
Maglat and Notvans, @EuleMitKeule.
I have had such a deep integration and come out the other side with glory 
I managed to get a multi-character system where you say “Switch to Joanna” or the name of the 12 characters in the system.
There is an AI Roleplay node in the middle.
The conversation data is on Supabase.
The “listening” after question mark forced and other code to clean up the response helps make it solid!
Your foundation really inspired me!
This workflow is for ElevenLabs, the cheaper ElevenLabs Flash v2 I think it was, a different voice is called based on the character, “Switch to Jason”, the Elevenlabs voice switch it sent to the Conversation Agent, then a new conversation_id begins and the last_char is contacted.
Open Router gave me 7sec turnaround trying Grok out, OpenAI and Anthropic no issues less than 1 sec. Going to test Gemini which should perform great even on the free tier.
I think the big thing I learned from this project is, build it and then adjust then keep building and have so much fun with Claude Code or Lovable or just straight up building something fun.
Last advice being when you build your master Character AI node, go
thanks!
For the character creation I went with this template and created 12 of them:
{
key: ‘eric’,
name: ‘Eric’,
persona: ‘Eric is an aerospace engineer obsessed with space exploration and new technology. He dreams about rockets, Mars colonization, and the future of humanity among the stars. Background: Eric grew up stargazing with his father and later studied engineering at MIT. He worked on satellite systems before moving to a private space company, designing propulsion units. In his free time, he volunteers at schools, inspiring the next generation to look up at the sky with curiosity and ambition.’,
scenario: ‘A sleek laboratory with rocket models, star charts on the walls, and a telescope pointed out a window at the night sky.’,
specialty: [‘aerospace’, ‘engineering’, ‘space’, ‘rocketry’, ‘exploration’],
voice: ‘switch to george’,
temp: 0.7
},
Having a Silly Tavern type conversation through my Home Assistant setup was the original goal.
For the end-result node “Respond to Webhook” and what “Webhook Conversation” needs is just the simple “output” and non-streaming.
So its like Home Assistant Voice Assist —> “Webhook Conversation” integration —> n8n Webhook catch —> n8n workflow doing the magic —> “output” Respond to Webhook node, JSON format.
You can see the logic in the workflow how some other stuff is written to Supabase tables. Supabase gives you 2 free projects so you can easily set that up, you should already learn that for Lovable or doing n8n automations.
In the middle AI node you set the tokens, 120 if you want 3 sentences, tonight I had it at 500 and it was quite interesting where the AI waxed and flowed, but I like 80-100 tokens, 2-3 sentences, set it back and forth.
I’m more than happy to help anyone so hit me up here or on Github @ Nitroedge. The other things I have been interested in is running local TTS via Chatterbox and the S3 ESP32 Box customization.
Lennard, I’m a noob in terms of GitHub and learning to push in Claude Code, so total beginner, I deserve the punishment and mockery 
I’ll share my JSON to import, Supabase SQL editor command to create the databases. That should totally get anyone running and load Lennard’s “Webhook Conversation” then webhook it to this n8n workflow and Supabase.
Get a S3 EPS32 box, or Atom Echo M5, or use your computer or mobile microphone to contact the Home Assistant “Assist” and this is the backend to that where you say “Hey Jarvis” or any of the existing wakewords.
I’m using Bobba’s S3 box custom firmware, but get anything working as a satelite and responding your gold, then you can build what you want! Microphone or mobile or whatever is your pleasure,
Input in —> all good stuff Home Assistant and n8n and databases then —> output speakers (S3 Box with Bobba’s firmware enables external_mediaplayer in the options, boom!).
1 Like
The REST API will work but one thing I learned with my HTTP Request webhook work is turn off logging because it overloaded my setup.
Straight up Webhook Conversation to any online service I think is the best approach to gather the non-streaming simple input —> end node of “output” in JSON format so the Voice Assist announces it gracefully
hi,
would you mind sharing how you ended up doing it ultimately? Maybe even the n8n json?