Get conversation inputs and outputs to dashboard

Configuration Voice Assistant
1

Hey guys, i try to populate my dashboard with notifications about my voice assistant / assist questions and responses.

My idea is to put up a notification on my dashboard, when i ask my voice assistant anything, but i don’t know how to get conversation inputs, intents, responses and so on to node red to pass it to some entity.

i can just monitor conversation.home_assistant (or any other agent) and the entity just changes its timestamp on activation.

is there a specific event i can filter for that holds this data or do i need to create an automation to get all incoming conversations?

Greetings :slight_smile:

1 Like
2

I would also like to find a solution for this. I want to listen to the agent’s response. I haven’t figured out how to do it yet.

3

Agreed, it’s impossible to debug what’s happening without this.

4

Nobody else interested in putting this kind of information on their dashboards?

It seems like there is already something out for the voice-assistant ESP32-S3-Box-3, but not available in HA dashboards?

5

If you are looking for a way to capture what the assistant heard and says then you can add the following to your voice assistant device yaml configuration. This gives you a text sensor within Home Assistant with the last conversation request and response.

text_sensor:
###### Text Request ######
  - platform: template
    id: text_request
    name: Text Request
    icon: mdi:message-processing
    internal: false
    on_value:
      lambda: |-
        if(id(text_request).state.length()>32) {
          std::string name = id(text_request).state.c_str();
          std::string truncated = esphome::str_truncate(name.c_str(),31);
          id(text_request).state = (truncated+"...").c_str();
        }


###### Text Response ######
  - platform: template
    id: text_response
    name: Text Response
    icon: mdi:message-reply-text
    internal: false
    on_value:
      lambda: |-
        if(id(text_response).state.length()>32) {
          std::string name = id(text_response).state.c_str();
          std::string truncated = esphome::str_truncate(name.c_str(),31);
          id(text_response).state = (truncated+"...").c_str();
        }


voice_assistant:
...
  on_listening:
    - lambda: id(text_request).publish_state("...");
    - lambda: id(text_response).publish_state("...");
  on_stt_end:
    - text_sensor.template.publish:
        id: text_request
        state: !lambda return x;
  on_tts_start:
    - text_sensor.template.publish:
        id: text_response
        state: !lambda return x;

Hope this helps you out.

6

Thank You very much!
I already found this solution and thought about why i have to send data so many times from server to satellite to server again, if i could just get everything on the server.

But your solution is at least a small workaround! :slight_smile:

Another Question is, if we can get all conversations in and outputs from assist.
Probably to act on anything written or said. Also to get conversations from smartphone and smartwatch apps.