I believe it’s prematurely reacting to an false positive object detection, which Frigate filters out within a few frames but this still sends a notification. This is based on the fun job of watching the debug Frigate feed and cross referencing that with notification failures.
More importantly I’m missing genuine notifications and I’ve no idea why as they’re in Frigate and they meet all the conditions. I’m trying to work out the HA traces but I haven’t the foggiest. Working on it now.
I think it is related to the cooldown function. I have it set to 30 seconds. I believe the cooldown timer is being used even though there is no notification was sent?
Perhaps, though I wouldn’t describe it as prematurely. If frigate sends an Mqtt message with a new event for the set camera, the automation triggers no matter what. It’s the only message we can really use to start the workflow.
If frigate later dumps the whole event then I’m not sure what we can do in this case, though I’m not saying it’s impossible to try.
Correct, it’s based on the automation triggering, not the outcome. If you know a way to check if a notification was sent I’d genuinely appreciate assistance implementing that instead.
Would it be possible to replace the notification message with the response from generative AI?
I already have a blueprint that can save the snapshot to a local file so I can call the service on that snapshot. Just worried about timing as it takes a second or two to receive the response back.
Thank you for the awesome blueprint. I am experiencing an issue where I am unable to view links to live feeds or video clips in notifications on my iOS device, despite snapshots links displaying correctly. Interestingly, the notifications, including dynamic snapshot updates, work flawlessly on my Apple Watch, suggesting the problem is isolated to the links sent to the iOS App Notification Actionable buttons.
This is the structure of links sent to the app buttons:
Apple Watch Model and watchOS Version: Series 7 v 10.3
Configuration and Access:
I have configured Frigate and Home Assistant for external access via Nginx. Both systems are accessible outside my local network, and I can view live streams and playback clips directly in the Frigate UI via Safari on my iOS device without any issues. If NGINX config between HA and Frigate is suspected I can share those site configs.
Issue Description:
When receiving Frigate notifications through the Home Assistant Mobile App on my iOS device, snapshots are correctly displayed when a snapshot button is clicked. This indicates successful integration for static image delivery through the Frigate API.
However, notifications containing links supposed to direct to live feeds or video clips fail to function on the iOS device. Tapping on these links open the iOS default browser and ends at a 404 error.
Contrastingly, the same notifications when viewed by raising my Apple Watch display snapshots perfectly in the preview, updating in real-time as better images are captured by Frigate. This successful operation on the watch but not on the phone suggests the issue may be specific to links sent to the buttons shown on iOS Home Assistant Mobile App Notifications.
Expected vs. Actual Behavior:
Expected: Notifications should allow clicking through to valid links of live feeds and video clips on all devices, including iOS smartphones, in a similar manner to how snapshots are displayed and updated correctly on Apple Watch as Frigate receives better snapshots.
Actual: While snapshots are linked correctly on both iOS and watchOS devices, links to video content from iOS Notifcation Buttons direct to invalid resources.
Troubleshooting and Observations:
The discrepancy in functionality between iOS and watchOS devices suggests the issue may lie in how the iOS Home Assistant Mobile App processes or displays notifications, particularly regarding video content links.
Network configuration and streaming protocol compatibility are unlikely to be the cause, given successful direct access via Safari to the Frigate Server UI with flawless operation off streaming, clips and snapshots in iOS Safari.
I believe this information points towards a potential issue in the Home Assistant Mobile App for iOS or the notification blueprint’s handling of video links. I hope this detailed report assists in diagnosing and ultimately resolving the issue. I am ready to provide further details, logs, or participate in additional troubleshooting as required.
Thank you again for your contribution to the community.
Also I imagine you would ask so, I have not made any manual tweaks to the HA Automation YAML. It is as generated by the Blueprint UI Setup:
alias: Front Door Person Notification
description: ""
use_blueprint:
path: SgtBatten/Stable.yaml
input:
camera: camera.front_doorbell
notify_device: 99cd10646a3ec05dXXXXXXXXXX75b87a
message: Person Detected - Front Door
critical: "true"
update_thumbnail: true
ios_live_view: true
base_url: https://www.mysite.net
tap_action: >-
{{base_url}}/api/camera_proxy_stream/camera.{{trigger.payload_json['after']['camera'].lower()}}?token={{state_attr(
'camera.' ~ camera, 'access_token')}}
url_1: "{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4"
Few days ago (7th Feb) my automation stop to work correctly.
The audio portion of the automation is it still wotk but not the video one.
alias: Alert from Frigate Ingresso
description: Audio and Video Alert from Frigate Ingresso
trigger:
- platform: state
entity_id:
- binary_sensor.zone_2_person_occupancy
from: "off"
to: "on"
condition: []
action:
- service: tts.cloud_say
data:
cache: false
entity_id: media_player.cucina
message: >-
Attenzione, un intruso all ingresso. Ripeto, un intruso è stato
avvistato all ingresso
- wait_template: "{{ is_state('media_player.cucina','idle') }}"
continue_on_timeout: true
- service: media_player.volume_set
data:
volume_level: 0.7
target:
entity_id: media_player.cucina
- service: media_player.play_media
data:
media_content_id: >
{{ "http://192.168.1.13:8123" + state_attr("camera.ingresso_person",
"entity_picture") }}
media_content_type: image/jpeg
target:
entity_id: media_player.cucina
- delay:
hours: 0
minutes: 0
seconds: 10
milliseconds: 0
- service: tts.cloud_say
data:
cache: false
entity_id: media_player.cucina
message: Ripeto, un intruso era stato avvistato all ingresso
- wait_template: "{{ is_state('media_player.cucina','idle') }}"
continue_on_timeout: true
- service: media_player.turn_off
data: {}
target:
entity_id: media_player.cucina
- service: tts.clear_cache
data: {}
mode: single
I don’t remember to have done any relevnt update.
This is the message in Register
Logger: homeassistant.components.automation.alert_from_frigate_ingresso
Source: components/automation/__init__.py:690
Integration: Automazione (documentation, issues)
First occurred: 7 febbraio 2024 alle ore 20:30:08 (51 occurrences)
Last logged: 10:25:02
Error while executing automation automation.alert_from_frigate_ingresso: Error rendering data template: TypeError: can only concatenate str (not "NoneType") to str
Hi @SgtBatten, it looks like the notification is sending outside of the specified hours/rule for group.persons being away or not. In my testing with the developer tools, it returns the string ‘false’, but it still sends out a notification anyway when it is triggered. Below is my check:
{{'true' if now().hour in [21,22,23,24,1,2,3,4,5,6] or is_state('group.persons', 'away') else 'false'}}
Is there some other setting that I may be missing? The above checker is in the field: Custom Filter (Optional - Advanced)
Do I need to toggle a specific radio button to be on? There are several in the filters section, but I couldn’t tell which one needed to be toggled on to enable the custom filter, if so.
Perhaps to avoid bloat, and keep your GPT prompt in one place, GPT calls should be done from a script, which can be triggered in the SgtBatten blueprint by “Custom Action - Auto Trigger”. Then the original notification will be independent of the potentially slower GPT call. I’d imagine that {{camera_name}} and/or the snapshot URL could be sent through the script call data, but haven’t tried it.
To go further, the blueprint could be updated to optionally use a script to replace the notification message with the response variable from a script. Obviously that would add a point of possible failure/delay.
For reference, my current Google Gemini notification script (independent of the SgtBatten blueprint):
Camera Analysis - Gemini
alias: Camera Analysis - Gemini Pro
sequence:
- service: camera.snapshot
metadata: {}
data:
filename: /config/www/temp_cam_capture.jpg
target:
entity_id: "{{input_camera}}"
- service: google_generative_ai_conversation.generate_content
metadata: {}
data:
image_filename: /config/www/temp_cam_capture.jpg
prompt: >-
This is a security camera frame from my yard. Please concisely describe
anything out of the ordinary in the scene, such as people, animals,
apparently moving vehicles, disturbances, damage, and any other
potentially interesting or suspicious features. On the other hand, if
the scene appears placid, then just briefly describe it. There is no
need to mention absolutely fixed features like a house or street in the
background, a closed gate, cars in their proper parking spot, or patio
furniture. You can also mention ostensibly normal, but potentially
interesting things, e.g. small animals, an open gate, a vehicle with
lights on.
response_variable: response
- alias: echo
if:
- condition: template
value_template: "{{\"echo\" in notify}}"
- condition: template
value_template: "{{target_echos is defined}}"
then:
- service: notify.alexa_media
data:
message: "{{ response.text | trim(' ') }}"
data:
type: announce
target: "{{target_echos}}"
- if:
- condition: template
value_template: "{{\"phones\" in notify}}"
then:
- service: notify.all_phones
data:
title: Gemini
message: "{{response.text | trim(' ')}}"
alias: phones
mode: queued
max: 3
icon: mdi:brain
Yes you can theoretically replace the message. Simply set it such that it displays a normal message or the generated message if it exists. So if the updated message gets created after a second or two, and you have updates enabled, the next snapshot should come through with the new message.
I gave you bad information last time sorry. Custom filter acutally does need a boolean, not a string. So change ‘true’ to True and ‘false’ to False, inlcuding removing the apostrohpes
Do you think the clips not being in the correct place could be a Frigate recording configuration issue? I don’t think so since I can access the clips via Frigate UI but I’m curious just because there are so many different ways to configure Frigate itself.
I don’t think that’s the issue. Seems access related. Have you got allow unauthenticated access ticked in the frigate integration within home assistant?
My vanilla installation of the blueprint, with what I feel is a generic setup creates Clip links which include the camera name: https://www.mysite.net/api/frigate/notifications/170XXXXXX7.7XXXX3-u8hvz5/CAMERA_NAME/clip.mp4?external_auth=1
But the Figate docs say that a clip link does not have the camera in it, it has the same root url as a snapshot or a thumbnail.
Removing the camera name yields the identical result for me. Not sure if the API url changes to simplify it or not, but either way, both endpoints are currently valid. If you remove the camera name you get the same issue I imagine?
What if you remove the last query about external auth?
Yes same result without the camera, or removing the external auth marker after clip.mp4. If you use both root url structures can you view clips on your own installation? I guess I am asking because they both result in 404 for me and I am curious if one but not the other works on an installation known to work with your stock blueprint. Frigate documentation might also be wrong.
