Klar! NLU - Assist alternative with multi-intent, follow-ups, 60+ languages

lbentivegni · August 19, 2026, 11:32pm

Hi everyone,

We built Klar NLU, a local conversation engine for Home Assistant Assist. House commands stay on your machine - no cloud, no model weights for the NLU itself. An optional LLM can chat or polish the spoken reply; it does not turn lights on similiar to Assist. We just were annoyed a little bit / why should an LLM get the full context? why can´t just the NLU turn on two lights etc.

It’s free, open source (MIT), and we don’t sell a subscription.

What it does

Turns spoken household sentences into HA intents: lights, climate, covers, locks, fans, media (including Music Assistant), timers, lists, scenes
Follow-ups like “turn it off” keep the last target
Clarifies when a device is ambiguous
Personality in HA (butler, Jarvis, pirate, …) for the voice just for fun
Optional LLM fallback for chit-chat when it isn’t a home command / less context faster for local AI use

In our Benchmark, Klar hit 100% vs ~31% for built-in Assist templates. More details in our docs

ngl. a lot of AI was used but that should be clear for everybody looking into the NLU topic.

Status

Still a few bugs sometimes

Get it

GitHub: GitHub - FABBricate-IT-Solutions/klar-ha-nlu: Deterministic NLU for Home Assistant. 67 languages, fully local. Assist alternative: 100% vs 31% on 9,922 DE/EN tests. · GitHub
HACS: add that repo as Integration, download Klar NLU, restart, then set the Assist pipeline conversation engine to Klar NLU

Expose the devices you want to talk to, try five phrases, and if something misses use Mapping in the UI.

Have fun talking to the house - we’d love to hear how it goes.

eriri · August 20, 2026, 12:49pm

Hi, can you be clearer in your documentation about what are the roles of the HACS integration and the app/addon? Do we need both? Does one improve the other? I have no idea whether do I install one or the other or both because there is no clear instructions in the installation documentation.

lbentivegni · August 20, 2026, 1:52pm

Thanks for calling this out.. that gap was on us. Klar is still a very new project, so some of the install docs were thinner than they should have been. We just rewrote that section.

Short version: Klar is two pieces. They do different jobs. Installing both does not make parsing more accurate.

HACS integration conversation agent for Assist (syncs rooms/devices, runs intents). You always need this if Assist should use Klar.
App (add-on) runs the NLU engine in its own container and serves Mapping / Lab (sidebar Klar NLU).
Bundled engine the same integration can download the GitHub binary and start the engine inside Core if you have no App. (example HAOS)

not yet tried to install the Container Image on a separate Host etc. But this should also work you then need to configure some sort of token. Recommended from our Side is the App Addon.

What to install

Home Assistant OS: both. HACS for Assist, the App for the engine and Mapping/Lab. At setup pick Use the Klar NLU App or Docker.
No Supervisor (Container / Core): HACS only → Start the bundled engine (HACS only). Assist works; Mapping/Lab are not in the sidebar.
App without HACS: the UI can open, Assist will not use Klar.

Do not run the App and the bundled engine at the same time.

User docs are something we will keep improving. Feel free to contribute - issues and small docs PRs are very welcome.

famewolf · August 26, 2026, 11:10pm

@lbentivegni I have klar installed and it appears to be working as the conversation agent..it turns things on and off however I can find no way to get it to tell me my upcoming calendar events. Something the LLM CAN do and if klar can’t I would much rather it pass it on rather then telling me to tell it what light to turn off. Can you implement an “Ask the llm” feature so if klar can’t handle it we could tell it “Ask the llm ”.

Every time I ask it anything calendar related like “What’s on my calendar” or “what are my upcoming events” I get “Create Calendar Event is off” as my reply. I added aliases of “my calendar” and “my upcoming events” to the calendar I have exposed. You talk about mapping but I don’t see anything about mapping? Also I don’t speak or read german and that’s all the app will show. I even selected english in the settings and clicked save. I also recall reading it tied in with music assistant but I tried “resume music”, “play depeche mode” and a few other phrases none of which it understood.

mag2000 · August 28, 2026, 1:34pm

I installed the app (HAOS) and the integration as well. Now I’m having the following problem: the integration can’t connect to the app because the app is assigned a hostname with a prefix (xyz-klar-nlu). I can change the hostname in the integration, but it’s always overwritten with “klar-nlu”. so no connection possible.

lbentivegni · August 29, 2026, 1:00pm

@famewolf thanks for the report. All of that should be fixed in the latest release (2026.8.53). Please update both the HACS integration and the App so they match.

Calendar list / create / delete / move is in now, so “What’s on my calendar” and “what are my upcoming events” should work. The “Create Calendar Event is off” reply was a bug.

Just as a note - To send a phrase to the LLM yourself, use a policy in the operator UI:

Settings → Devices & services → Klar NLU → Configure → set Conversation agent for chit-chat to your LLM. Keep the Assist pipeline conversation engine on Klar NLU.
Open the App sidebar Klar NLU (not the Lovelace Klar card) → Rules → Policies.
Click Rule.
When → Phrase: the words you want to say, for example Ask the LLM (4–200 characters).
Then: LLM prompt.
In the box, write the system prompt for the fallback agent (required). Example: Answer this in the user’s language. Do not control devices.
Save.

First matching rule wins. After that, saying “Ask the LLM …” (or whatever phrase you set) goes to that agent with your prompt. You can add more rules the same way, e.g. Phrase what’s on my calendar → LLM prompt. I know i need to document more but there are still bugs which are currently a little bit more important.

Mapping is House → Mapping in that same App. The operator UI follows the Home Assistant language, so English should stick after the update. Music Assistant phrases like “play Depeche Mode” and “resume music” should parse as well.

If anything still misses after the update, send the exact phrase.

lbentivegni · August 29, 2026, 1:07pm

@mag2000 that hostname overwrite was a bug. The latest release (2026.8.53) keeps Supervisor-prefixed hosts like xyz-klar-nlu instead of rewriting them to klar-nlu.

Update the HACS integration (and the App to the same version). The URL you set should stay.

famewolf · August 29, 2026, 4:26pm

Thanks for the follow up but I’m a little confused because I’m currently on 2026.8.55 which unless you had a typo is newer than that. Additionally, it still shows up all in German, so if you could say specifically how to get it to change the EnglishThat would be very helpful. I’m on HACS and I uninstalled the app when it was all in German, so right now I’m only running the integration. If I add the app back, is it going to mess up anything?

lbentivegni · August 29, 2026, 6:58pm

@framewolf You’re right.. sorry, I saw that right after I posted. 2026.8.55 is newer. Not a typo. We had another language-config bug after that, and 55 still has it. The fix is 2026.8.61. Update HACS to that and restart.

There are actually three language settings (I had to check them myself):

Integration UI — should now follow your Home Assistant language.
Pipeline language — I made all languages visible to cut config effort. That was a mistake.
Operator UI — under Settings, if you use the App. You don’t have to. It’s for troubleshooting and a bit more customization.

Uninstalling is fine. Almost everything comes from HA. The only thing you might lose is custom policies you set in the Operator UI.

I’m also looking at another small regression (volume vs light scoring very close) and some calendar create bugs. List works.

I’ll switch my own setup to staging once main feels somewhat usable.

Thanks again for testing.

famewolf · August 29, 2026, 8:28pm

Got it going and set up the Good Morning and Good Night routines to trigger the appropriate script. Seemed to work well. App shows in english now. Is there something somewhere or can I query a list of comands that Klar will respond to? I’ll admit to being a relative newbie to Home Assistant so just trying to get a feel for all it can do.

lbentivegni · August 29, 2026, 10:45pm

Klar’s job is: talk to it like a person. You should not have to memorize a command grammar. Throw a normal sentence at it. If it is sure, it runs the house. If it is not sure, or the action is risky, it asks first instead of guessing.

It is a local, rule-based voice engine for Home Assistant, an alternative to the normal Assist behaviour. It is not a cloud assistant and not an LLM with tools. Klar turns speech into device actions. An optional LLM in Home Assistant can chat or polish the spoken reply. It does not get to flip switches.

How you could talk to it

Name a room, a device, or both. Follow-ups work on the same conversation: “turn it off”, “a bit dimmer”, “and the kitchen too”.

I would rate German and English as the strongest packs. They are the ones we live-test hardest, also thanks to your feedbacks. Other Assist locales are compiled too and take the same path. Because Klar is built around a home graph, aliases, multi-intent, and follow-ups (not a giant list of sentence templates), those other languages should still feel a lot better than stock Assist. Home Assistant’s approach is different: custom sentence triggers can get you far there as well, but you have to write and maintain a lot more of that yourself.

Lights

You say	What should happen
Turn on the living room lights / Licht im Wohnzimmer an	Lights on in that room
Turn off the kitchen and the hallway	Two rooms, one sentence
Living room and kitchen lights on / Wohnzimmer und Küche an	Same, multi-target
Living room lights at 100% / Wohnzimmerlicht auf 100%	Absolute brightness
Set the living room lights to 30 percent	Absolute brightness
A bit dimmer / ein bisschen dunkler	Relative step down (after a living-room target)
Brighter / Wohnzimmer hell / volle Helligkeit	Relative up, or “full” as 100%
Medium / mittelhell	Named level (about 50%)
Make it red / wohnzimmer rot	Color, if the light can do it
und die Küche auch / and the kitchen too	Same action, other room

Climate

You say	What should happen
How warm is it in the living room? / Wie warm ist es im Wohnzimmer?	Read temperature
Set the living room heat to 21 / Heizung Wohnzimmer auf 21	Setpoint
Turn on the living room heat, then “to 21”	Follow-up fills the number

Covers, garage, fans

You say	What should happen
Open the living room blinds / Rollo im Wohnzimmer auf	Open
Close the living room blinds / Rollo im Wohnzimmer zu	Close. Confirm first (risky)
Set the garage door to 40% / Garagentor auf 40 Prozent	Position
Set the living room blinds to 75	Position

Locks and other critical actions

You say	What should happen
Unlock the front door / Wohnungstür aufschließen	Guard: “Really?” Then yes / ja
Lock the front door / Wohnungstür abschließen	Same confirm
nein	Cancels. Nothing moves

Unlocking, locking, and closing covers are treated as risky. The plan stays in the session until you confirm. A vague “open the door” should not silently fire.

Music and TV

You say	What should happen
Play Queen / Spiel Queen	Music Assistant search-and-play
Play Queen in the kitchen / Spiel Queen in der Küche	That room’s player
Play music in the kitchen / Musik in der Küche	Resume / start in that room
Play the album Rumours by Fleetwood Mac in the living room	Search with type hints
What’s playing? / Was läuft?	Now playing on the last player
Make it quieter / etwas leiser	Relative volume
Set the volume in the living room to 30	Absolute volume
Next track / pause / nächster Titel / Pause	Transport on the last player
Turn on the living room TV / Wohnzimmer Fernseher an	That TV, not the lights

Klar does not invent artists or playlists. Music Assistant has to resolve the search. Alexa as a speaker is a player/integration problem, not a Klar parse miss. If Alexa does not accept Music Assistant playback, Klar can still understand “play Queen” and you can still hear nothing on that box.

Calendar, clock, weather

You say	What should happen
What time is it? / Wie spät ist es?	Clock
What’s the weather? / Wie wird das Wetter?	Weather
What’s on the calendar? / Was steht im Kalender?	List events
What do I have tomorrow? / Was steht morgen an?	Tomorrow
Add dentist tomorrow at 3 to my calendar / Trage den Termin Zahnarzt morgen um 15 Uhr in den Kalender ein	Create event
Delete dentist / move dentist to Friday	After a list, follow-ups stay in HA

Known wobble: event titles sometimes pick up leftover words. “Add the event Name tomorrow at 3 to my calendar” is safer than a very loose sentence. Guards / clarify should kick in when the title looks wrong. We are still tightening this.

Timers, lists, scenes, queries

You say	What should happen
Start a timer for 10 minutes / Timer 10 Minuten	Timer
Cancel the timer / Timer abbrechen	Cancel
Add milk to the list / Milch auf die Liste	Shopping / todo
Milk done	Check it off
Activate All off / Aktiviere Alles aus	Scene
Are the living room lights on? / Ist das Licht im Wohnzimmer an?	State query

Multi-intent (one breath, several actions)

Turn off the living room lights and set heat to 21
Mach das Licht im Wohnzimmer aus und Heizung auf 21
Turn on the living room and kitchen lights
Wohnzimmer und Küche an
Turn off the living room lights and lock the front door
(lights run; lock still needs confirm)

When it is unsure: guards

Klar is allowed to not act.

Clarify: two lamps in one room, or a nickname that matches more than one device. “Which one?”
Confirm: locks, unlocks, closing covers, and other risky steps. Answer yes / ja on the same conversation.
Reject / hand off: “What’s the capital of France?” is not a home command. “Tell me a joke” / Erzähl einen Witz can go to an optional chat LLM if you configured one. The LLM still does not run devices. (we might add a feature flag for it)

If a device is missing, it is usually not exposed (Settings → Voice assistants → Expose), not a language bug. Nicknames live in Klar NLU → House → Mapping (or HA entity aliases).

What already works well

Everyday lighting, heat, covers, fans, scenes, timers, lists
Several devices or rooms in one sentence
Absolute and relative dim / volume (“100%” vs “quieter” / “less bright”)
Follow-ups in one conversation (“and the kitchen too”, “to 21”, “pause”)
Calendar list / create / delete / move (titles still sometimes sloppy)
Critical actions stop for a spoken confirm
Music Assistant search-and-play (“play X”, albums, radio, queue) on a player that can actually search
Optional personalities and LLM reply polish. Device control stays with Klar
On-device, no cloud in the engine

Limitations (honest)

It still mishears or mis-binds sometimes. The design is: ask, don’t guess. If a guard does not fire, that is a bug we want.
Speech-to-text is not Klar. If Assist heard the wrong words, the parse never sees what you meant.
Calendar create can glue extra words onto the title. Prefer “add the event Name tomorrow at 3”.
Music needs a searchable player (Music Assistant). Alexa Echo playback is outside Klar. The sentence can be correct and the speaker still silent.
No general knowledge. It will not cook, plan a trip, or invent a playlist that is not in the library.
Only exposed entities. Hidden sensors and infra are invisible on purpose.
A room with several generic “lights” gets a question, not a random lamp.
Generated language packs are thinner than German/English. Same engine, less live-hardened phrasing.
There is no per-home learning yet. Your nicknames help (Mapping). Your speaking style is not trained into a private model.

How we improve it today

We do not fine-tune a neural net on your house.

We keep datasets: real household sentences. Next to them sit oracles: the expected detection (intent + slots) and the expected action (which light goes to 30%, which door must confirm). A change is only “good” if those oracles still pass.

That is why a new phrasing (“and the kitchen too”, “a bit dimmer”) lands as a test case first, then as engine behavior. Motivation is the same pattern you want as a user: say it naturally, get a precise, checkable action.

Where this is going

The next step is a more generic learning mode:

Point Klar at a house (your rooms, your names, your habits).
Optionally configure an LLM as a teacher, not as the thing that presses buttons.
Let it propose policies for that home: aliases, “when I say X I mean Y”, safer defaults.
Keep Klar as the executor. The model may suggest a rule. It does not unlock the door on its own.

Until that exists, Mapping + oracles are the way a home gets sharper. Speak normally, let the guards catch the misses, and treat leftover calendar words or a silent Alexa as known edges, not as “you used the wrong command.”

Hope this helps and have fun. And Thanks again for the Feedback

famewolf · August 30, 2026, 5:34am

Thank you SO MUCH for that detailed reply which answered so many of my questions before I even asked them. I also want to take a moment to just say thank you for providing KLAR to us and making it open source. I look forward to putting it through it’s paces. I’m gonna have to be careful though as I have automations and blueprints which handle checking/adding events to calendar, reminders and a few other things. I’m going to have to disable them while I try the new options. For music in music assistant I did try “play infinite library playlist in living room” intending for it to play on the stereo in there. It replied “done” but no music ever started up. I probably messed up somewhere just reporting it for feedback. Also I saw an option named “nlu_rag”. I know with the llm rag is used to reduce the amount of tokens used. Is this somethign we want on?

I have several devices named “Coffee Machine Power”, “Instant Pot Power” etc. If I tell it to “turn on the instant pot” will it catch that or do I need to say power? I plan to test this in the morning (well later morning as it’s 1:30am) and have something cooking overnight in the instant pot and in morning will make coffee so it will get tested then.

Also for adding events to calendar does it prompt for information or do I have to put it all on the line ie “add event to calendar, appointment with doctor tomorrow at 2 pm”. I’ll test that as soon as I get the existing automations, scripts etc disabled. Feel free to ignore all this rambling. Just excited about all the potential and I realize I’m asking way to much.

famewolf · August 30, 2026, 8:47pm

Some things are still in german as you’ll see in the screenshot.

famewolf · September 1, 2026, 9:15pm

My understanding is klar handles what it can handle but when it gets a question it doesn’t understand it passes it to the LLM if you’ve configured one. If this is accurate, then it’s not happening. I’ve asked various questions the LLM could answer that klar could not and it just returns various errors. If I’m misunderstanding please let me know.

What’s the weather and What’s coming up are two off top of my head.

lbentivegni · September 2, 2026, 5:34pm

@famewolf
The UI language issue you reported earlier is fixed.

Your read is close, but Klar does not send every leftover question to the LLM. There are two separate LLM features, and both use the same chit-chat agent.

Set Settings → Devices & services → Klar NLU → Configure → Conversation agent for chit-chat. On that agent (OpenAI, Ollama, Gemma, …), turn Assist tools / Control Home Assistant off. If tools stay on, Klar skips the LLM. That is the safest setup, especially with a small local model.

Fallback is for smalltalk (jokes, stories, greetings) and for speech Klar did not classify as a home command. That is the only path where the model answers on its own.

Refine is off by default. Turn on Let the LLM refine NLU replies. After Klar parses and Home Assistant runs the intent, the model only rewrites the already-spoken NLU line into the selected personality, one or two sentences. It does not take over the command, invent a forecast, or answer a question Klar already handled. If refine fails, the short Klar reply stays.

Weather and calendar are home commands, so they stay with Klar. They do not go to fallback. Refine only restyles the result after Klar has a real answer.

What’s the weather looks for a weather.* entity exposed to Assist (Met.no, OpenWeather, and similar). If that entity is there, Klar reads it. If it is not, English replies “I do not see a weather entity.” The LLM is not asked.

What’s coming up is not in the calendar lexicon, so it is leftover speech and should hit fallback once the agent above is set. For the calendar itself, use “What’s upcoming?”, “What’s on the calendar?”, or “What’s on the calendar tomorrow?” Those need a calendar entity exposed to Assist. Without one you get “No calendar is available.”

So: expose a weather entity for weather, use a calendar phrase plus an exposed calendar for the agenda, use fallback only for things Klar did not treat as a house command, and use refine only to change how a successful Klar reply sounds.

famewolf · September 5, 2026, 12:30am

@lbentivegni Thank you. Something had reset my connection to home assistant’s conversation agent instead of klar which explains why alot wasn’t working that should have. I do note when I ask “What’s on my calendar tomorrow” I get a weather report and not a calendar report. I tried it in “Lab” under the klar app but not sure this is helpful:

{
  "schema_version": "2.0",
  "text": "What's on my calendar tomorrow?",
  "conversation_id": "432355f7-ded1-4738-bafc-4278f308191b",
  "decision": {
    "type": "execute"
  },
  "speech": "",
  "confidence": 0.9407500000000001,
  "margin": 1,
  "selected_candidate_id": "plan-000",
  "candidates": [
    {
      "id": "plan-000",
      "plan": {
        "confidence": 0.9407500000000001,
        "margin": 1,
        "evidence": [
          {
            "kind": "action",
            "source": "lexicon_exact_question",
            "value": "get_state",
            "score": 0.96,
            "exact": true
          },
          {
            "kind": "binding",
            "source": "calendar",
            "value": "bound",
            "score": 0.85,
            "exact": false
          }
        ],
        "steps": [
          {
            "index": 0,
            "intent": {
              "name": "KlarGetCalendarEvents",
              "slots": [
                {
                  "name": "domain",
                  "value": "calendar"
                },
                {
                  "name": "day",
                  "value": "tomorrow"
                }
              ]
            },
            "confidence": 0.9407500000000001,
            "evidence": [
              {
                "kind": "action",
                "source": "lexicon_exact_question",
                "value": "get_state",
                "score": 0.96,
                "exact": true
              },
              {
                "kind": "binding",
                "source": "calendar",
                "value": "bound",
                "score": 0.85,
                "exact": false
              }
            ]
          }
        ]
      },
      "score": 0.9407500000000001,
      "margin": 1,
      "policy": "calendar",
      "precedence": 1,
      "evidence": [
        {
          "kind": "action",
          "source": "lexicon_exact_question",
          "value": "get_state",
          "score": 0.96,
          "exact": true
        },
        {
          "kind": "binding",
          "source": "calendar",
          "value": "bound",
          "score": 0.85,
          "exact": false
        }
      ]
    }
  ],
  "plan": {
    "confidence": 0.9407500000000001,
    "margin": 1,
    "evidence": [
      {
        "kind": "action",
        "source": "lexicon_exact_question",
        "value": "get_state",
        "score": 0.96,
        "exact": true
      },
      {
        "kind": "binding",
        "source": "calendar",
        "value": "bound",
        "score": 0.85,
        "exact": false
      }
    ],
    "steps": [
      {
        "index": 0,
        "intent": {
          "name": "KlarGetCalendarEvents",
          "slots": [
            {
              "name": "domain",
              "value": "calendar"
            },
            {
              "name": "day",
              "value": "tomorrow"
            }
          ]
        },
        "confidence": 0.9407500000000001,
        "evidence": [
          {
            "kind": "action",
            "source": "lexicon_exact_question",
            "value": "get_state",
            "score": 0.96,
            "exact": true
          },
          {
            "kind": "binding",
            "source": "calendar",
            "value": "bound",
            "score": 0.85,
            "exact": false
          }
        ]
      }
    ]
  },
  "evidence": [
    {
      "kind": "action",
      "source": "lexicon_exact_question",
      "value": "get_state",
      "score": 0.96,
      "exact": true
    },
    {
      "kind": "binding",
      "source": "calendar",
      "value": "bound",
      "score": 0.85,
      "exact": false
    },
    {
      "kind": "preferred_area",
      "source": "satellite",
      "value": "bedroom",
      "score": 1,
      "exact": true
    }
  ],
  "trace": {
    "stages": [
      {
        "stage": "normalize",
        "duration_us": 30,
        "detail": "4 normalized tokens"
      },
      {
        "stage": "features",
        "duration_us": 7,
        "detail": "1 clauses"
      },
      {
        "stage": "action_candidates",
        "duration_us": 2,
        "detail": "1 action hypotheses"
      },
      {
        "stage": "target_resolution",
        "duration_us": 22968,
        "detail": "1 scored targets"
      },
      {
        "stage": "binding",
        "duration_us": 0,
        "detail": "1 complete candidate plans"
      },
      {
        "stage": "ranking",
        "duration_us": 1455,
        "detail": "1 ranked candidates"
      },
      {
        "stage": "safety_decision",
        "duration_us": 3,
        "detail": "execute"
      },
      {
        "stage": "planning",
        "duration_us": 0,
        "detail": "1 executable plan steps"
      }
    ],
    "discarded": [],
    "tokens": [
      "what",
      "on",
      "calendar",
      "tomorrow"
    ],
    "normalized": "what on calendar tomorrow"
  },
  "briefing": false,
  "policy_trace": {
    "compiled_risky": false
  }
}

Additional followup: “What’s the temperature outside?” results in "Thermostat is 76 degrees C. Problem 1. I asked for outside. Problem 2. It’s not 76 degrees C it’s F (for me in USA).

lbentivegni · September 6, 2026, 4:12pm

This is a heads-up. Stable today is still 2026.9.3. The next CalVer from staging is a large product cut: Assist speech, LLM, and settings move into the Klar engine / operator UI.

Upgrade engine and integration together. A mixed pair will fail. On HAOS, update the App and HACS in the same sitting, then restart Core.

Breaking changes (read before you install)

Assist LLM is no longer a Home Assistant conversation agent. Chat, refine, stories, and calendar talk use an OpenAI-compatible endpoint you set in Klar (Settings → LLM): local Lemonade/Gemma, or any /v1 server. The old “pick an HA LLM agent and Klar will call it via the OpenAI SDK” path is gone. If you only had a fallback agent configured in the integration and never set an engine LLM, chat/refine will stay silent until you configure LLM in the Klar UI.
Personality, refine, quiet ack, RAG, and voice live in the Klar operator UI (and on the engine), not in the Home Assistant options form. Leftover HA options are used only if the engine has no settings yet. After upgrade, open the App → Settings and confirm Voice + LLM. The Personality entity on the Klar device still patches the engine.
Python Assist fallbacks are gone. Refine prompts, fallback/RAG prompts, and post-execute speech are built in the Rust engine. If the App/engine is old and HACS is new (or the reverse), those routes fail closed. You will not get a silent Python rebuild.
POST /api/parse is still gone (since V2, already in 2026.9.3). Only POST /api/v2/parse. Mentioned again because third-party scripts that still hit /api/parse will keep failing.
LLM refine is hold-until-accept. Off by default. When on, a rewrite is spoken only if the engine accepts it. Weather/climate/status lines that the model would have emptied or turned into “ist .” stay on the original Klar sentence.
First start may show the setup wizard (six steps, skippable). Details below. Light/dark theme persists.
Reinstalling the App still wipes container options (token, ingress panel). There is now Settings → Backup (download/restore operator settings; secrets optional). Use that before you uninstall/rebuild.

Setup wizard

The App (sidebar Klar NLU) is the operator console: Settings, House, Lab, Rules. Lovelace Klar is only the last Assist turn. The wizard is not a seventh tab. It is a first-run overlay. You can Skip, press Escape, or click the backdrop. Whatever you already filled in is saved. Re-open it anytime from Settings.

Six steps:

This screen. Console language and theme. Assist can still use all language packs.
Units. Metric or imperial for temperatures only (parse and speech). Lights, covers, and timers stay as they are.
Engine LLM. Optional OpenAI-compatible URL, model, key, thinking flag. Leave empty to finish without an LLM and set it later in Settings. If you save a working endpoint here, LLM refine is turned on for you.
Voice. Personality (including Jarvis or a custom voice). Stored on the engine. Home Assistant Configure does not need these fields.
House settings. Quiet ack, NLU-RAG, Assist tools on chat, and the rest of the product toggles. Same store as Settings.
How to use Klar. Detected install path (App / Docker / bundled / sample house), then five check phrases for Assist.

Do not set an LLM as the Assist conversation engine. Voice and the LLM stay in this console.

Lotse

Lotse is the operator sidebar in the App, not a voice assistant and not the Assist pipeline. It is a trainer/guide for this console (Settings, House, Lab, Rules). You can resize the pane. Light theme persists.

Lotse uses the same engine LLM as Assist chat and refine. No extra Home Assistant OpenAI/Ollama integration. Until you tap Allow, it writes nothing. After consent it can apply tools on the current lane (match, language, house). Replies stay locked to the product. You should never see raw LOTSE_VIEW: markup in the pane.

If the LLM is not configured, Lotse stays empty. Set Settings → LLM first (or step 3 of the wizard).

What changes for Assist (household)

Spoken status uses a live HA snapshot, not a stale graph. Weather and climate speak the real temperature (“27.5 Grad”) instead of an empty “ist .”.
Floor / whole-home / room status stay a room list. A climate or weather entity in the snapshot no longer collapses “Wohnung” or “Wohnzimmer” to a single temperature. Named devices in a room/floor are still spoken (as in 2026.9.x).
Metric vs imperial (temperatures only). We now support imperial as well as metric - thanks @framewolf for your report. Settings and the setup wizard have a metric / imperial toggle. Default is metric. Existing installs keep speaking and setting Celsius. Imperial treats a bare number as Fahrenheit (set the heat to 70 becomes 21 °C in Home Assistant) and speaks weather/climate in °F. An explicit scale on the phrase (70 fahrenheit, 21 Grad) still wins. This does not change other units (covers, volume, and so on).
Calendar no longer double-speaks after a streamed answer. Household weather is skipped only when a calendar noun is present.
Streaming of LLM tokens into Assist chat and TTS (already started in 2026.8.67–2026.9.0) is kept. Listen closes after execute.
Allow Assist tools on chat stays off by default. When on, after Klar parse the model may call Core Assist tools (2026.9 names like intent__HassTurnOn). Device commands still go through Klar first.
Personalities include Jarvis and a custom voice (name, seed prompt, 0–10 sliders). Refine follows the language of the line, not a forced German wrap.

Operator UI (App / Mapping / Lab)

New Settings layout: LLM (presets, model list, thinking flags), Voice, Backup, metric / imperial temperature units.
First-run wizard (above). Home charts on by default.
Lotse: resizable operator sidebar (above).
Rules path grid (match / language / house) and trainer lanes (consent, validate, govern seeds). This is the ADR 0001 work that was held on staging.
Overlay match on/off and lexicon set paths from the UI.
Lab stays the Assist plan. Pipeline settings are visible.

What is not new vs 2026.9.3

Already on stable, still true:

Debian security updates in the runtime image
Named-device room/floor speech
Opt-in Assist tools on chat
LLM token streaming into Assist
V2 parse contract

How we will show notes in Home Assistant

HACS will show the GitHub release body before install, with Breaking Changes at the top when a cut has any. The Supervisor App changelog was empty because CHANGELOG.md was not next to the add-on config. That is fixed on the next cut so the App update dialog has notes too.

famewolf · September 6, 2026, 7:34pm

This was produced with ai assistance so apologies if any of it is inaccurate or unclear.

Klar NLU issue: custom phrases silently stop working after restart (memory-only store)

Summary

On 2026-09-06, all custom phrases (e.g. “Good Morning” → HassTurnOn →
script.wake_up_trigger) stopped executing. Every utterance fell through to
decision: chat with confidence 1.0 and no binding, so the voice assistant
just greeted the user back instead of running anything. The phrases were still
visible in the Klar web UI, but the engine had none loaded.

Root cause as far as I can determine: custom phrases live in engine memory
only and are wiped by any restart, with nothing on disk to reload them from.

Environment

Home Assistant (HAOS VM under Proxmox): core 2026.9.x
Klar integration + engine: 2026.9.1 (stable:2026.9.1 in
klar_nlu/version, manifest 2026.9.1, HACS
FABBricate-IT-Solutions/klar-ha-nlu installed 2026.9.1, commit ae10883)
Engine serves HTTP on 127.0.0.1:10520, Wyoming on 127.0.0.1:10500

Versions / upgrades checked

The engine binary and web UI on my box were both replaced 2026-09-04
18:07 EDT (mtime of /homeassistant/klar_nlu/klar and
/homeassistant/klar_nlu/ui/assets/*), consistent with the 2026.9.1 update.
I found no upgrade on 2026-09-05. So the phrase loss does not correlate with
an upgrade replacing files — it correlates with restarts (below).

What was blank, and what the UI still showed

At failure time, with the UI still displaying the phrase rows (“Good Morning”
→ HassTurnOn → script.wake_up_trigger, “Good Night” → …):

GET /api/custom → []
GET /api/bundle/entries → {"enabled": false, "count": 0, "bytes": 0, "entries": []}
GET /api/bundle/dataset → []
The on-disk config /homeassistant/klar_nlu/klar_nlu.json contains no
phrases either ("aliases": {}, "custom": []).

So the UI showed phrases the engine did not have. That display appears to be
stale (pre-restart tab) rather than live engine state.

Meanwhile conversations.jsonl logged every attempt as e.g.:

{"tokens": ["good", "morning"], "decision": "chat", "confidence": 1.0, ...}

with empty evidence — the correct outcome for an engine with nothing to match
against, but bewildering when the UI shows the phrase as configured.

Why they were gone: restarts wipe the in-memory store

The binary is launched with --data-dir /homeassistant/klar_nlu, and that
directory contains no phrase/bundle store file (only the binary, token, UI,
klar_nlu.json, and conversations.jsonl). I searched the whole config dir:
no file anywhere contains the custom phrases. Conclusion: they are memory-only.

Home Assistant restarted three times that morning (07:34, 07:53, 12:58 EDT;
the 12:58 restart precedes the 13:48 voice failure by ~50 min). Each restart
respawns the engine binary, discarding the phrases. A host power loss later
that day (13:49) and the subsequent reboot repeated the wipe.

Related aggravator: a phrase like “turn on wake up” can also fail with
decision: clarify (“What should I turn on?”) because scene.wake_up and
script.wake_up_trigger share the name “Wake up” — only an explicit custom
phrase disambiguates, which makes the volatility above hurt more.

Workaround I implemented (all on-box, no external host)

klar_phrases.json in the config dir: the 6 phrases as the exact JSON
POST /api/custom accepts (tracked in my local git repo as source of truth).
shell_command.klar_push_phrases: curls the file into the engine using the
token from /homeassistant/klar_nlu/token.
script.klar_restore_phrases (90 s boot grace, manually runnable) +
automation.system_restore_klar_phrases_on_restart (fires on the HA start
event) to re-push after restarts.

Verified: after re-push, all phrases parse-test at decision: execute,
confidence 1.0.

Ask

Persist custom phrases to disk (e.g. in the --data-dir) and reload
them on engine start. Memory-only storage for user configuration is the
bug — a restart should never silently unconfigure voice control.
If the web UI cannot reach a live engine, it should say so (or show the
store as empty) instead of displaying stale phrase rows.
Consider logging a warning at startup when the custom-phrase store is
empty, so the failure is visible instead of surfacing as mysterious
decision: chat fallbacks.

Update 2026-09-06 — upcoming CalVer cut (staging → stable)

Per lbentivegni’s announcement, the next cut moves Assist speech/LLM/settings
into the engine + operator UI, with breaking changes. Notes relevant to this
report, all on 2026.9.1 unless stated:

The diagnosis above is unaffected; filing against 2026.9.1 as observed.
The new Settings → Backup (operator settings download/restore) may or
may not cover custom phrases — if it does, that is the author’s blessed
persistence story and this workaround can retire. Please confirm whether
custom phrases are included in that backup.
My workaround POSTs the phrase list to /api/custom. The Rules rework
(match/language/house grids) may move or reshape that endpoint — I will
re-verify after upgrading and adapt. Flagging here so the endpoint contract
is on your radar.
Noted and appreciated: mixed App/engine pairs now fail closed rather than
silently rebuilding, and POST /api/parse staying gone (I already use
/api/v2/parse).
I will upgrade engine + integration together with a Core restart in one
sitting per the instructions, and will re-test all phrases after.