If anyone ever ends up in the unfortunate position to have to look over a relative with early onset short term memory loss, will know that keeping an eye on them when they are on their own is critical to their safety.
My mother over the past few months has suffered a mini stroke that has left her with partial short term memory loss. While carers visit her I wanted some peace of mind that she will be OK when she’s left alone in the house.
So I’ve turned to frigate and home assistant to help. I’ve installed a 10+ tapo C110 cameras throughout the property (one in each room) plus some mmWave sensors with the cameras linked to a frigate instance running on a NAS that I’ve left at her house. The mmWave sensors report directly to my HA instance which is used to report all of this. Her house and mine are connected via Tailscale .
The video feeds are processed to check for people presence and record events accordingly. However the stock automations left a bit to be desired, as they are geared towards generating alerts and events when someone is there, not when they are not. Also I could not integrate at a glance other detection devices (like the mmWave sensors for areas where video feeds are not the right thing to do).
So I made the below:
- A blueprint that scans all the people presence detection entities exposed by frigate in HA and creates a list of occupied rooms in a text box. This can be then displayed as a “chip” in my HA dashboard. This way at a glance I can see where in the house she is currently, when I check in on her, instead of having to scan through the 10+ camera feeds that are available (and also to retain an element of her privacy).
- The above blueprint also exports a text field containing the room where a person was last seen. This is useful to know where she last was when the system stopped detecting her (eg. front door, staircase etc).
- A date/time stamp when the last person was seen.
- It can work beyond just cameras - mmWave presence detection devices and motion sensors can also be used as inputs for areas that are more sensitive to her privacy (and of course there is no image there)
Basically with the above three, I can at a glance with three chips, know if she’s around and if not, where she last was and when.
(in the above I also have a list of picture entities of high criticality areas (staircase, kitchen, door) that show the last captured person snapshot, so I can check for example that she hasn’t fallen down the stairs or that she hasn’t left the house, at a glance)
However acting quickly is also important. So I setup another blueprint that
- Generates a mobile notification when no-one is in the house and a configurable amount of time has passed.
- The notification attaches a screenshot of the last place she was seen, the name of the room/camera and the time.
This is useful to setup alerts so if for example she leaves the house or similar, and she hasn’t returned for a set amount of time, I can call her and check in on her.
I’m currently testing this and tweaking it but wanted to share in case anyone else was in a similar position to me and finds this useful.
I’d welcome any suggestions on how to improve this further! I’m reasonably new to coding in HA (but have plenty of experience in C++ etc) so even this has been a steep learning curve.
Things I’d love to be able to do, but I haven’t figured it out yet:
- Fall detection (and immediate notification)
- Scream/shout detection (unfortunately for some reason frigate doesn’t recognise these even though the audio recognition is enabled and these are available as events).
Helpers: (plus one input boolean configured in the UI used to control the notifications so I dont get duplicates)
input_text:
occupied_rooms_cameras:
name: "Occupied Rooms (Cameras)"
initial: ""
max: 255
last_occupied_room_camerars:
name: "Last Occupied Room (Cameras)"
initial: "Waiting for first detection..."
max: 255
input_datetime:
last_detection_timestamp_cameras:
name: "Last Detection Timestamp (Cameras)"
has_date: true
has_time: true
rest_command:
fetch_latest_frigate_event:
url: "http://IP:9150/api/events?limit=1&has_snapshot=1"
method: GET
headers:
accept: "application/json"
timeout: 10
content_type: "application/json"
payload: "{}"
Occupied locations blueprint
blueprint:
name: Occupied Rooms Update with Last Detected Room and last detection timestamp
description: >
This blueprint updates a given input text field with a coma separated list of names of the rooms where
occupancy has been detected using the specified sensors and corresponding room names. It is set to "" (empty) if no presence is
currently detected. \n\n
It also updates another input text field with the last room where presence was detected and the
date/time when the last presense was detected. \n\n
Finally when presence is detected, it resets the Notification Sent Boolean helper. That helper can be used to enable single issuance of
notifications when presence has not been detected or other automations.
domain: automation
input:
sensors:
name: Room Occupancy Sensors
description: The binary sensors detecting presence.
selector:
entity:
domain: binary_sensor
multiple: true
names:
name: Room Names
description: The names of the rooms corresponding to each sensor. Used to populate the outputs.
selector:
text:
multiple: true
occupied_rooms_text:
name: Occupied Rooms
description: The input_text entity where the list of occupied rooms will be stored, separated by a coma (,). Field will be empty if no rooms are currently occupied.
selector:
entity:
domain: input_text
last_detected_room_text:
name: Last Occupied Room
description: The input_text entity where the name of the room where presence was last detected will be stored.
selector:
entity:
domain: input_text
last_seen_datetime:
name: Last Seen Date Time
description: The input_datetime entity where the date and time of the last presence detection will be stored.
selector:
entity:
domain: input_datetime
notification_sent_boolean_in:
name: Notification Sent Boolean
description: The input_boolean entity used to track whether a notification has been sent. It will be turned off when presence is detected.
selector:
entity:
domain: input_boolean
variables:
sensors_list: !input sensors
names_list: !input names
trigger:
- platform: state
entity_id: !input sensors
from: "on"
to: "off"
- platform: state
entity_id: !input sensors
from: "off"
to: "on"
condition: []
action:
- service: input_text.set_value
data:
entity_id: !input occupied_rooms_text
value: >
{% set ns = namespace(occupied_rooms=[]) %}
{% for i in range(sensors_list | length) %}
{% set sensor = sensors_list[i] %}
{% set name = names_list[i] %}
{% if is_state(sensor, 'on') %}
{% set ns.occupied_rooms = ns.occupied_rooms + [name] %}
{% endif %}
{% endfor %}
{% if ns.occupied_rooms | length > 0 %}
People detected in the {{ ns.occupied_rooms | join(', ') }}.
{% else %}
{% endif %}
- service: input_text.set_value
data:
entity_id: !input last_detected_room_text
value: >
{% set ns = namespace(last_room='') %}
{% for i in range(sensors_list | length) %}
{% set sensor = sensors_list[i] %}
{% set name = names_list[i] %}
{% if (trigger.entity_id == sensor) %}
{% set ns.last_room = name %}
{% endif %}
{% endfor %}
{% for i in range(sensors_list | length) %}
{% set sensor = sensors_list[i] %}
{% set name = names_list[i] %}
{% if is_state(sensor, 'on') %}
{% set ns.last_room = name %}
{% endif %}
{% endfor %}
{{ ns.last_room }}
- service: input_datetime.set_datetime
data:
entity_id: !input last_seen_datetime
datetime: "{{ now().strftime('%Y-%m-%d %H:%M:%S') }}"
- service: input_boolean.turn_off
target:
entity_id: !input notification_sent_boolean_in
Notification blueprint:
blueprint:
name: Notify on Absence of Presence Detection
description: >
This blueprint monitors the last detection timestamp and sends a notification
to a specified mobile device if no presence has been detected anywhere in the property for a configurable
amount of time. The notification will include the last detected room, the last detection time, and a thumbnail of the latest event.
domain: automation
input:
last_detection_timestamp_in:
name: Last Detection Timestamp
description: The input_datetime entity where the timestamp containing the last presence detection is stored.
selector:
entity:
domain: input_datetime
last_occupied_room_text_in:
name: Last Occupied Room Text
description: The input_text entity where the name of the last occupied room is stored. This will be used in the notification text.
selector:
entity:
domain: input_text
time_threshold_in:
name: Time Threshold (minutes)
description: The amount of time (in minutes) without presence detection before sending a notification.
default: 60
selector:
number:
min: 1
max: 1440
unit_of_measurement: minutes
notify_device:
name: Device to Notify
description: The device to send a push notification to. The device needs to run the official Home Assistant app to receive notifications.
selector:
device:
integration: mobile_app
notification_sent_boolean_in:
name: Notification Sent Boolean
description: The input_boolean entity used to track whether a notification has been sent. Reset this when presence has been detected to enable notifications again.
selector:
entity:
domain: input_boolean
occupied_rooms_text:
name: Occupied Rooms
description: The input_text entity where the list of occupied rooms will be stored. Used to check that no rooms are occupied, as the timestamp only contains the last detection time, not if any rooms are currently occupied.
selector:
entity:
domain: input_text
server_address_in:
name: Homeassistant Public Address
description: The public facing http(s) address of homeassistant, used to display notification snapshots.
selector:
text:
multiline: false
client_id_in:
name: The frigate client ID
description: If using multiple frigate instances, specify the client ID for the server used in this automation.
selector:
text:
multiline: false
rest_command_in:
name: Rest Command
description: The rest_command service name that will be used to fetch the latest Frigate event from the corresponding frigate server.
selector:
text:
multiline: false
url_when_clicked:
name: The url to navigate to when clicked
description: Enter the dashboard uri to open when the notification is clicked.
selector:
text:
multiline: false
variables:
last_detection_timestamp: !input last_detection_timestamp_in
last_occupied_room_text: !input last_occupied_room_text_in
time_threshold: !input time_threshold_in
notification_sent_boolean: !input notification_sent_boolean_in
occupied_rooms_text: !input occupied_rooms_text
server_address: !input server_address_in
client_id: !input client_id_in
trigger:
- platform: time_pattern
seconds: "/30"
condition:
- condition: template
value_template: >-
{% set last_time = state_attr(last_detection_timestamp, 'timestamp') | default(0, true) | as_datetime %}
{% set threshold = time_threshold | int %}
{% set has_notification_been_sent = states(notification_sent_boolean) %}
{% set occupied_rooms = states(occupied_rooms_text) %}
{{ (now() - last_time).total_seconds() > (threshold * 60) and has_notification_been_sent == 'off' and (occupied_rooms == '' or occupied_rooms == 'unknown' or occupied_rooms == 'None') }}
action:
- action: !input rest_command_in
response_variable: frigate_response
- domain: mobile_app
type: notify
device_id: !input notify_device
title: "No Presence Detected"
message: >
{% set last_room = states(last_occupied_room_text) %}
{% set last_time = states(last_detection_timestamp) %}
{% if last_time in ['unknown', 'None', ''] %}
No recent presence detected.
{% else %}
{% set last_dt = strptime(last_time, '%Y-%m-%d %H:%M:%S') %}
{% set today = now().date() %}
{% set yesterday = (now() - timedelta(days=1)).date() %}
{% set formatted_date =
"today" if last_dt.date() == today else
"yesterday" if last_dt.date() == yesterday else
last_dt.strftime('%d %b') %}
{% set formatted_time = last_dt.strftime('%H:%M') %}
Last presence detected in the {{ last_room }} {{ formatted_date }} at {{ formatted_time }}.
{% endif %}
data:
image: "{{ server_address }}/api/frigate/{{ client_id }}/notifications/{{ frigate_response['content'][0]['id'] }}/snapshot.jpg"
url: !input url_when_clicked
- service: input_boolean.turn_on
target:
entity_id: !input notification_sent_boolean_in
mode: single