I’m trying to develop a household security check based on this video.
I took his basic example and modified it for my own house. As a standalone script, it works to generate a response for each of the 4 fields depending on which one I select. What I cannot do is figure out how to get it to return a voice announcement of the response when I say “Okay Nabu, run a security check”.
I’ve tried an automation that captures the sentence that I speak in a variable, passes it to the script, which then runs the appropriate action, producing a response with multiple variables. That bit works. The issue I have is getting the script to pass the right response variable back to the automation and having the automation wait until that response is available before proceeding. The camera snapshots part of the script can take 70s to return a response.
The author seems to suggest that simply exposing the script itself to Assist is enough to make the magic happen, but I’m struggling with that too. Custom sentences and Intent Scripts seem to be beyond me currently.
Hers’s part of the script and the response it produces when I run it:
alias: Security Check
sequence:
- choose:
- conditions:
- condition: template
value_template: >-
{{ security_query == 'Security Check | Query Motion and Door
Contact Sensors' }}
sequence:
- delay:
seconds: 2
- variables:
response:
query_all_motion_sensors: |
{% set sensors = {
'binary_sensor.garage_occupied': '(Garage)',
'binary_sensor.living_room_occupied': '(Living Room)',
'binary_sensor.kitchen_occupied': '(Kitchen)',
'binary_sensor.utility_room_occupied': '(Utility Room)',
'binary_sensor.cloakroom_occupied': '(Cloakroom)',
'binary_sensor.dining_room_occupied': '(Dining Room)',
'binary_sensor.hallway_occupied': '(Hallway)',
'binary_sensor.landing_occupied': '(Landing)',
'binary_sensor.office_occupied': '(Office)',
'binary_sensor.guest_bedroom_occupied': '(Spare Room)',
'binary_sensor.seans_room_occupied': '(Seans Room)',
'binary_sensor.jamies_room_occupied': '(Jamies Room)',
'binary_sensor.master_bedroom_occupied': '(Our Bedroom)',
} %} {% for sensor, note in sensors.items() %} {{
states[sensor].name if states[sensor] is not none else sensor
}}: {{ 'detected' if states[sensor].state == 'on' else 'clear'
if states[sensor].state == 'off' else 'unknown' }} {{ note }}
|{% endfor %} Be brief and do not list all entities, instead,
only mention sensors that are not clear. Do not include emoji
and make sure your response is conversational since it is
being audibly broadcasted over a speaker system.
query_all_contact_sensors: |
{% set sensors = {
'binary_sensor.front_door_sense_contact': '(Front Door)',
'binary_sensor.garage_door_left_contact': '(Garage Door Left)',
'binary_sensor.garage_door_right_contact': '(Garage Door Right)',
'binary_sensor.back_door_sensor_contact': '(Back Door)',
'binary_sensor.side_door_sensor_contact': '(Side Door)',
'binary_sensor.downstairs_windows': '(Downstairs Windows)',
'binary_sensor.upstairs_windows': '(Upstairs Windows)'
} %} Alarm State: {{ states('alarm_control_panel.downstairs')
| title }} | {% for sensor, note in sensors.items() %} {{
states[sensor].name if states[sensor] is not none else sensor
}}: {{ 'open' if states[sensor].state == 'on' else 'closed' if
states[sensor].state == 'off' else 'unknown' }} {{ note }} |
{% endfor %} Only mention doors/windows that are open. Keep it
conversational as this will be audibly broadcasted.
- stop: Alright. Im done.
response_variable: response
alias: Query All Motion and Contact Sensors
fields:
security_query:
selector:
select:
options:
- General Security Check | Query Everything
- Security Check | Query Motion and Door Contact Sensors
- Security Check | Query Camera Snapshots
- Security Check | Query Garage Camera Snapshot and Door Sensors
multiple: false
description: >-
Select a query to determine the current status of motion sensors, whether
or not they are detecting motion, and/or request an analysis of various
security cameras in the house. This will analyse the most recent snapshot
from cameras and provide this analysis to you.
name: Security Query
required: true
default: General Security Check | Query Everything
description: >-
Use this tool to query motion sensors, contact sensors, and camera snapshots
in and around the home to give home occupants feedback on motion in and
outside the home as well as provide an analysis of what the security cameras
are seeing in the house. Do not include emoji in your response and make sure
your response is conversational since it is being audibly broadcasted over a
speaker system.
icon: mdi:security-network
mode: parallel
max: 10
If I run just that part it returns:
query_all_motion_sensors: |-
garage_occupied: clear (Garage)
| living_room_occupied: detected (Living Room)
| kitchen_occupied: clear (Kitchen)
| utility_room_occupied: clear (Utility Room)
| cloakroom_occupied: clear (Cloakroom)
| dining_room_occupied: clear (Dining Room)
| hallway_occupied: clear (Hallway)
| landing_occupied: clear (Landing)
| office_occupied: detected (Office)
| guest_bedroom_occupied: clear (Spare Room)
| seans_room_occupied: clear (Seans Room)
| jamies_room_occupied: clear (Jamies Room)
| master_bedroom_occupied: clear (Our Bedroom)
| Be brief and do not list all entities, instead,
only mention sensors that are not clear. Do not include emoji
and make sure your response is conversational since it is
being audibly broadcasted over a speaker system.
query_all_contact_sensors: |-
Alarm State: Disarmed | Front Door Sense Door: closed (Front Door) |
Garage Door Left: closed (Garage Door Left) |
Garage Door Right: closed (Garage Door Right) |
Back Door Sensor Door: closed (Back Door) |
Side Door Sensor Door: closed (Side Door) |
Downstairs Windows: closed (Downstairs Windows) |
Upstairs Windows: open (Upstairs Windows) |
Only mention doors/windows that are open. Keep it
conversational as this will be audibly broadcasted.
This is where I’m at with the automation:
alias: Check the house is secure via Voice
description: >-
Runs a Security Check based on voice command and announces the result when
ready.
triggers:
- command:
- Run a security check
- Check the house security
- Perform a security check
- Check for motion
- Check the doors
- Check the security cameras
- Check the garage security
trigger: conversation
actions:
- variables:
query_map:
Run a security check.: General Security Check | Query Everything
Check the house security.: General Security Check | Query Everything
Perform a security check.: General Security Check | Query Everything
Check for motion.: Security Check | Query Motion and Door Contact Sensors
Check the doors.: Security Check | Query Motion and Door Contact Sensors
Check the security cameras.: Security Check | Query Camera Snapshots
Check the garage security.: Security Check | Query Garage Camera Snapshot and Door Sensors
selected_query: >
{{ query_map[trigger.sentence] | default('General Security Check | Query
Everything') }}
media_player_entity: >
{% set assist_device = trigger.device_id %} {% set entities =
device_entities(assist_device) %} {% set media_player_entity = entities
| select('search', 'media_player.') | first %} {{ media_player_entity }}
- action: persistent_notification.create
data:
title: Media Player ID Debug
message: "Media Player Response: {{ media_player_entity }}"
- action: persistent_notification.create
data:
title: Security Query Debug
message: "Security Query: {{ selected_query }}"
- data:
security_query: "{{ selected_query }}"
action: script.security_check
response_variable: response
- variables:
response: |
{% if response.query_all_contact_sensors != '' %}
{{ response.query_all_contact_sensors }}
{% else %}
Nothing to report
{% endif %}
- action: persistent_notification.create
data:
title: Script Response Debug
message: "Script Response: {{ response | default('no response') }}"
- wait_template: "{{ response }}"
timeout: "00:00:30"
continue_on_timeout: true
enabled: false
- variables:
security_response: "{{ state_attr('script.security_check', 'response') | default('') }}"
error_message: |-
{% if state_attr('script.security_check', 'last_triggered') is none %}
The security check script did not run.
{% elif state_attr('script.security_check', 'response') is none %}
The security check script did not return a response.
{% else %}
An unknown issue occurred.
{% endif %}
enabled: false
- choose:
- conditions:
- condition: template
value_template: "{{ security_response | length > 0 }}"
sequence:
- data:
entity_id: "{{ media_player_entity | string }}"
language: en-GB
options:
voice: AlfieNeural
message: "{{ security_response }}"
action: tts.cloud_say
default:
- action: persistent_notification.create
data:
title: Security Check Failed
message: "⚠️ Security Check failed: {{ error_message }}"
notification_id: security_check_error
- action: tts.cloud_say
data:
entity_id: "{{ media_player_entity | string}}"
language: en-GB
options:
voice: AlfieNeural
message: >-
There was a problem running the security check. {{ error_message }}
Would you like me to try again?
- wait_for_trigger:
- command:
- "Yes"
- Try again
- "No"
- Cancel
trigger: conversation
timeout: "00:00:30"
continue_on_timeout: false
- variables:
retry_response: "{{ wait.trigger.event.data.command | default('No') }}"
- choose:
- conditions:
- condition: template
value_template: "{{ retry_response in ['Yes', 'Try again'] }}"
sequence:
- target:
entity_id: automation.check_the_house_is_secure_via_voice
action: homeassistant.turn_on
data: {}
- conditions:
- condition: template
value_template: "{{ retry_response in ['No', 'Cancel'] }}"
sequence:
- data:
entity_id: "{{ media_player_entity | string }}"
language: en-GB
options:
voice: AlfieNeural
message: Okay, I won't try again.
action: tts.cloud_say
enabled: false
mode: single
It’s a work in progress with a few debug actions to check the variables that are being returned. Some work, some don’t.
I think that due to the way the script structures each response, in order to get the right response in the automation I have to do something like this:
- data:
security_query: "{{ selected_query }}"
action: script.security_check
response_variable: response
- variables:
response: |
{% if response.query_all_contact_sensors != '' %}
{{ response.query_all_contact_sensors }}
{% else %}
Nothing to report
{% endif %}
but I’d need a statement for every possible response variable and an action for each response.
I’d love to get it working with just a script that does all the legwork then passes the response to my speech engine script for announcement, but stuff like this in the response:
| Be brief and do not list all entities, instead,
only mention sensors that are not clear. Do not include emoji
and make sure your response is conversational since it is
being audibly broadcasted over a speaker system.
is beyond me. How is this bit even getting to the LLM as a prompt to process this response?
I think the author of the original script is on this forum, so I’m hoping (but not expecting) a steer in the right direction. I’m reasonably competent with scripts, automations, Jinja and variables. It’s the Assist bit with custom sentences in config and intent scripts that I’m struggling with (despite reading the documentation many times). So I just need a steer in the right direction.
In the meantime I’ll keep going and maybe have a Eureka! moment.
Thanks