🎤 GLaSSIST v2.0.0 - Now Your Smart Home Can Actually Ask You Questions!

patryk0802 · August 27, 2025, 1:07pm

Hey everyone!

You know what always bugged me about AI assistants? They’re basically fancy voice-controlled keyboards. “Turn on lights” → lights turn on. Revolutionary stuff, right?

I always wanted my smart home to be more like… well, smart. Not just waiting for commands like a digital butler, but actually asking me what I want. Like “Hey, you just got home - want me to turn on the lights and start coffee?” and I just say “yeah” instead of barking orders at my house.

What GLaSSIST Is

Started a few months ago as a simple desktop voice assistant for Home Assistant (original post here). Sits in your tray, listens for wake words, processes commands through HA Assist. You know, the usual stuff.

But v2.0.0 is where it gets interesting…

Pro tip: Works best with AI conversation agents like Claude, GPT, or Gemini instead of the basic HA conversation. They understand context much better and can handle more complex requests.

The Revolution: Interactive Prompts

Now Home Assistant can send questions TO GLaSSIST and get voice responses back. This changes everything:

Door opens → HA: “Welcome home! Turn on lights and start coffee?”
You: “Yes please!”
Result: Lights on, coffee brewing, no clicking anything

High energy usage detected → HA: “Want me to optimize power usage?”
You: “Do it”
Result: Climate adjusted, non-essential devices off

It’s like having an actual conversation with your house instead of just shouting commands at it.

How It Works (Simple Version)

HA detects something (motion, door, high usage, whatever)
HA asks GLaSSIST to ask you a question via HTTP API
GLaSSIST plays TTS
You respond with voice
HA AI processes your response and does the thing

No complex scripts needed - AI automatically understands context and executes appropriate actions.

Downloads & Setup

Windows: GLaSSIST-Setup.exe
GitHub: GitHub - SmolinskiP/GLaSSIST: Desktop voice-assistant application for Home Assistant with visual animations and VAD (Voice Activity Detection).

(Linux support exists but is janky - focusing on Windows for now)

Setup: Download → Run → Enter HA details → Pick wake words → Done.

What’s New in v2.0.0

Interactive Prompts API - The main event
Fixed wake word cross-activation - Basic models no longer trigger each other
ESC key support - Instantly hide interface during gaming
Better TTS handling - No more wrong audio playback
Context cleanup - Conversations don’t loop anymore

For the Nerds

Tech Stack:

Python backend with asyncio WebSocket for HA API
HTTP server (port 8766) for receiving HA prompts
Three.js frontend with audio visualization
WebRTC VAD for speech detection
100+ OpenWakeWord ONNX models

Integration:
Add to your HA configuration.yaml:

rest_command:
  glassist_prompt:
    url: "http://YOUR_GLASSIST_IP:8766/prompt"
    method: POST
    headers:
      Content-Type: "application/json"
    payload: |
      {
        "message": "{{ message }}",
        "context": "{{ context }}",
        "timeout": {{ timeout | default(15) }}
      }

Full setup guide with examples: INTERACTIVE_PROMPTS_SETUP.md

Fast example:

Intelligent Door Entry

  - alias: "🚪 Smart Welcome Home"
    description: "Context-aware arrival with personalized greeting"
    trigger:
      - platform: state
        entity_id: lock.front_door
        to: 'unlocked'
    condition:
      - condition: state
        entity_id: person.john
        state: 'home'
        for: "00:00:30"
    action:
      - service: rest_command.glassist_prompt
        data:
          message: >
            {% set time_away = (now() - states.person.john.last_changed).total_seconds() / 3600 %}
            {% if time_away > 8 %}
            Welcome back! You've been away for {{ time_away | round(1) }} hours.
            Shall I activate arrival mode? Lights on, climate to comfort, and check what happened while you were gone?
            {% else %}
            Quick trip! Want me to resume where we left off?
            {% endif %}
          context: "User just arrived home after being away. If they want arrival mode, turn on lights, set comfortable temperature, and provide status update of what happened while away"
          timeout: 25

Finally, a smart home that actually feels smart instead of just voice-controlled!

Feedback welcome - especially if you find creative ways to annoy your house with questions!

danw1 · August 27, 2025, 1:28pm

This legend is back!

This will work wonders for my housemate who leaves the oven on all the time! Needing to confirm you want it left on will be perfect!

Will have a run through and see how it goes, but installed and its working!

patryk0802 · August 27, 2025, 6:08pm

Love to hear that ;D The volume thing you asked i will do what i can to make it in this week

Orbernator · August 29, 2025, 4:24am

Hey! I have a Raspberry Pi and a webcam with a mic. Is there a kiosk mode or somthing along those lines coming down the line? Would be really good to have like a perminant GLaSSIST terminal somwhere in my house.

patryk0802 · August 29, 2025, 6:53am

Hm, i can do some tweaks, GLaSSIST on RPI sounds like a really good idea.

I will take my Raspberry and do some tests at this weekend, but i’m little afraid my code can be little heavy for that device.

What model of Raspberry do you have?
Do you have GUI installed or just terminal?

gjuillot · August 30, 2025, 12:08pm

Hi,

Your work seems amazing. I am trying to make it work but the connection to my local Home Assistant fails with in the logs

2025-08-30 13:39:36,360 - haassist - INFO - Connecting to Home Assistant: wss://homeassistant.local:8123/api/websocket
2025-08-30 13:39:36,480 - haassist - ERROR - Connection error: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1028)

I am new to Home Assistant so maybe I am doing something wrong…
I am running HA in a VirtualBox on the same computer as GLaSSIST (Windows 10)

patryk0802 · August 31, 2025, 8:26pm

Can you try change server address to your VM IP, not dns?

For example:
192.168.0.5:8123?

Over here:

Also try to update your homeassistant instance to newest version if possible. If it will not work i will add some more code to debug this problem deeper for you.

Update:
If i set my GLaSSIST to homeassistant.local:8123 it’s also not working, so setting IP address should resolve the problem 99%

2025-08-31 22:36:15,283 - haassist - INFO - Connecting to Home Assistant: wss://homeassistant.local:8123/api/websocket
2025-08-31 22:36:18,026 - haassist - ERROR - Connection error: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1028)
2025-08-31 22:36:18,026 - haassist - ERROR - Connection test: Cannot establish connection

danw1 · September 2, 2025, 9:18am

So i was thinking about this one, is there any way you can run yaml or an automation on start and finish?

Ie on wake word you can run an automation or yaml to save and lower volumes, and on close of return message you can then run another automation or yaml to return them?

I’m using this to broadcast notifications when there is someone walking up my driveway, and this lowers any music, tells me there is someone coming, then returns the music (though it does still wait for me to respond which i dont believe you can turn off, but i can deal with that). I do love a good broadcast that doesnt kill any music thats playing
Not sure if getting into that detail will effect your performance, because its working quite nicely currently

patryk0802 · September 2, 2025, 10:05am

About “wait for me to respond” i can add paramater to service call something like “wait_for_response: true/false”, honestly i don’t see usage for that but if you see no problem, it will be very easy to add i assume

About lowering volumes i think i don’t understand. You want to lower volume down of your PC or speakers?

If PC, it has to be in app, not in automations/yamls. Of course, you can have browser mod or something like this for controlling your PC from HA but assuming user have X HA addons for GLaSSIST to work properly is i think bad developing route ;p

If you wan’t to lower volume of speakers i see problems here too. You can have 0, 1, 5 speakers and you have to specify exactly what entities/speakers you want to volume down / mute. So you have to specify them by hand, then application must check volume level, lower it, do it’s job then restore. It sounds totally technically possible, but i have to check performance. At start of writing this comment i was very sceptic about this but when i’m writing this words it starts to look like a good idea I will try to add some option in settings for specifing entities to volume down at wake word activation.

About volume at broadcasting - you can do it in automation, something like this:

description: ""
mode: single
triggers: []
conditions: []
actions:
  - action: media_player.volume_down
    metadata: {}
    data: {}
    target:
      entity_id: media_player.patryk0802_s_2nd_echo_dot
  - action: rest_command.glassist_prompt_tablet
    data:
      message: Message to broadcast
      context: Context
      timeout: 15
      wait_for_response: false # TODO
  - delay: #fixed time needed for broadcasting message
      hours: 0
      minutes: 0
      seconds: 10
      milliseconds: 0
  - action: media_player.volume_up
    metadata: {}
    data: {}
    target:
      entity_id: media_player.patryk0802_s_2nd_echo_dot

danw1 · September 2, 2025, 10:37am

Wait for response would be an amazing feature for myself; not sure how other people use it. So i have multiple windows based touchscreen “dashboards” around my house. The one in the back patio is almost always playing music (through squeezlinte now as it works seamlessly) so i can use automations to control the volume of the music if it needs to broadcast something important (ie someone is breaking into your car… sad face) and i dont need to respond to ha, i just need to walk out the front and say g’day.
Side note - im not sure if anyone else would find it useful, but it is the only way to broadcast to a windows PC using your pre defined assistants voice (im not pedantic but i always hated having my announcements using tts have a diferent voice) and it doesnt stop the music from playing (if i was just useing browser_mod to play music and then broadcasted it would stop the song and not resume)

Now becuase i use squeezelite for my music, i can lower the volume from yaml (or almost any other device you can “shhh” from yaml) so my speaker doesnt interfear with my mic when making requests. This would be the same for VLC or browser_mod, so it would need to be able to lower volume prior to breadcasting

I came from a google home world, where voice was used for everything as well as the speaker being a music and broadcasting device… maybe im wanting too much from it?

patryk0802 · September 2, 2025, 11:01am

No no, you have great ideas, and every good idea is priceless I’m not a type of creative guy so every opinion is very important for me. Wait for response true/false is great idea to expand functionality with really low cost, but i would never think of that.

I’m just not sure how to implement this, i first time hear the word sqeezelite;p

How exactly you ‘shhh’ this devices? Are they media_player entities in HA? If not, then this can be little problematic. But maybe possible. Just explain this to me please

danw1 · September 2, 2025, 11:08am

Sorry, I explained poorly! They are controllable media entities. Same as browser mods media player, but a standalone exe which doesn’t destroy your browser cache.

example:

alias: Front Door Camera Popup on Motion
description: ""
triggers:
  - type: occupied
    device_id: 3363ff081c845f735c445e082fa7d8ab
    entity_id: d0af4aa9fe5a9d7d9ef4bc557e3f9327
    domain: binary_sensor
    trigger: device
conditions:
  - condition: state
    entity_id: automation.front_gate
    state: "on"
actions:
  - target:
      device_id:
        - browser_mod_9bff8737_faf3750b
        - browser_mod_f32b43bf_87a1cb0b
    data:
      title: Front Door Motion Detected
      content:
        type: picture-entity
        entity: camera.front_door_live_view
        camera_view: live
        show_name: false
        show_state: false
      size: wide
      dismissable: true
      autoclose: false
      timeout: 15000
      style: |
        --popup-min-width: min(90vw, 600px);
        --popup-max-width: 90vw;
        --popup-min-height: min(70vh, 400px);
        --popup-max-height: 70vh;
      browser_id:
        - 71500302e00c9ae031a97437f0b96346
        - ee9ace1e0ecf61dea06bba6154641156
    action: browser_mod.popup
  - variables:
      original_volume: >-
        {{ state_attr('media_player.squeezeplay_ac_67_5d_b7_ff_de',
        'volume_level') }}
  - action: media_player.volume_set
    data:
      volume_level: 0.1
    target:
      entity_id: media_player.squeezeplay_ac_67_5d_b7_ff_de
  - action: rest_command.glassist_prompt
    data:
      message: Motion at front door
      context: >-
        This is a notification only. Do not wait for response. No user input
        required.
      timeout: 1
  - data:
      entity_id: media_player.browser_mod_9bff8737_faf3750b
      message: Motion at front door
      language: en-in
    action: tts.google_translate_say
  - delay:
      seconds: 3
  - action: media_player.volume_set
    data:
      volume_level: "{{ original_volume }}"
    target:
      entity_id: media_player.squeezeplay_ac_67_5d_b7_ff_de
mode: single

edit: i still have one device that doesnt use GLaSSIST for notification, thats the TTS you can see there.
I use an agressive/abusive indian comedian as my voice assistant, thats why the TTS is en-in, just to clarify

patryk0802 · September 2, 2025, 11:24am

Ok, so if i make new tab in settings with:

Entities - where you can put any entities you want by names like “media_player.squeezeplay_ac_67_5d_b7_ff_de” for specific assistant
Target volume level after volume down - where you put this 0.1 or anything
some in-app mechanism to check volume level at start and restore volume level in entities after speech end

then it will work as expected i assume?

Should be pretty easy

danw1 · September 2, 2025, 11:36am

Yes, that exactly would suit me

patryk0802 · September 2, 2025, 6:37pm

Enjoy

Release # GLaSSIST v2.1.0 - Volume Management & TTS Enhancements · SmolinskiP/GLaSSIST

IMPORTANT NOTES FOR EVERYONE:

If you’re updating to GLaSSIST v2.1.0, you need to update your Home Assistant configuration.yaml:

OLD configuration (v2.0):

  rest_command:
    glassist_prompt:
      payload: |
        {
          "message": "{{ message }}",
          "context": "{{ context }}",
          "timeout": {{ timeout | default(15) }}
        }

NEW configuration (v2.1.0):

  rest_command:
    glassist_prompt:
      payload: |
        {
          "message": "{{ message }}",
          "context": "{{ context }}",
          "timeout": {{ timeout | default(15) }},
          "wait_for_response": {{ wait_for_response | default(true) | lower }}
        }

danw1 · September 4, 2025, 9:08am

This works way better than I expected! I forgot about TV background noise which it now also fixes!

It has created an issue where you can’t use voice to change your volume, but that’s kinda expected when you think it through. I’m sure that can be fixed with a helper + delayed automation and some conversation instruction.

Very impressive stuff, and you did that so fast!

Orbernator · September 9, 2025, 12:34am

Sorry for late response been bit flat out. I have a rasperry pi 5 which is my home assistant server but am planning on getting another. It will probably run RPiOS or Ubuntu

patryk0802 · September 10, 2025, 9:59am

Yes, you cannot change the volume of entities specified in the application because the application saves their previous volume level and restores it afterwards. While writing the code, I realized that this is probably how it will work, but I didn’t have a good idea how to solve this on the application side without hardcoding, which I would very much like to avoid ;).

But as you say, it should be achievable using the helpers and instructions included for AI in “context.”

I’d love to know if you managed to achieve the desired effect and, if so, how Or if you have any ideas on how to improve this in the application while avoiding the creation of rigid “ifs.”

danw1 · November 11, 2025, 6:08am

So I haven’t had a chance to sit down and play with volume yet, but wanted to stop by and say I use this every day. It’s replaced my Google home speakers and allowed me to keep all the functions with the addition of anything else I want.

I guess the next step is to port it to Android, given that’s the most common dashboards people use.

Well done

patryk0802 · November 20, 2025, 10:37am

I’m afraid porting to Android not gonna happen ;( First thing, i hate Kotlin/Java. Second - Android is too secure to allow things like custom WakeWord etc, so porting it can be even impossible.

Good to hear, i’m also started using it much more thanks to your upgrades