Use frigate and LLM Vision to send mobile notifications.
I decided to make this BP after having too many false positives with other blueprints with various triggers and quite often failed attempts at downloading the frigate files from the api for image and video analysis.
I have included a file check with the downloader integration before asking LLM vision to analyse the image or clip, if a failure occurs the automation will loop the actions and check again after a delay should the file download fail or not download in a specified time.
This is a work in progress and a automation i work on as and when i find errors and or improvements etc. so if you have any recommendations feel free to let me know and help if you can.
Configurable Inputs:
Frigate Cameras
Frigate Zones
Custom Zones
All Zones Required
Frigate Objects
Frigate Server address
LLM Vision Provider
LLM Vision Models (Common)
Custom Notification Message (image)
Custom Notification Title (image/video)
Enable LLM For Inital Notification
LLM Vision Prompt (image/video)
LLM Vision Target Pixels (image/video)
LLM Vision Remember (image/video)
LLM Vision Memory (image/video)
LLM Vision Tokens (image/video)
LLM Vision Creativity (image/video)
LLM Vision Frames (video)
LLM Vision Generate Title (image/video)
Downloader Directory
Downloader Sub Directory
Delay before snapshot analysis.
Delay before review clip analysis.
Wait for snapshot download.
Wait for review clip download
Companion App mobile devices
Click action of notification
Cool down: This allows a new and end event every 2 minutes for example.
Fixed parameters
Parallel automation with max set at 2.
Includes a logbook log event with quite a few details for debugging if required.
Requires:
- Home Assistant with an MQTT broker setup.
- Frigate integration with MQTT setup.
- LLM Vision integration with provider.
- Downloader integration.
I have only set this up for one Frigate camera, so you must use multiple instances of this automation for multiple cameras.
Thanks for contributing to the community with a new Blueprint.
I have a suggestion for you. Many people who are not familiar with directory structures will have problems installing this without the Home Assistant MY tools.
Adding a MY link for this Blueprint to your top post would help them a lot.
Here is the link to make that. Create a link – My Home Assistant
Note: if the original is in the forums here, only 1 code block can be in the top post ot the MY link tool will not work.
I can’t seem to get this to work and the traces seem to indicate that it fails on the download.
Does this require using the frigate add-on within haos, or can it use a separate frigate instance? The frigate server config input shows a default port of 8123, which is the ha default port. My frigate server is on a separate server and uses the default frigate port of 8971. Would it not also need a frigate username and password to login to the frigate server? I am not familiar with how the api works. Or, am I supposed to put in the ha server IP here?
Thanks for your work.
EDIT: I got this to work by putting my HomeAssistant server address into the frigate server configuration box. I used https with my internal FQDN and port 8123.
Hi Bobby in my instance i am using the HA frigate addon so its all on the same machine so the default value i have entered in the BP was based off my setup, although you have it sorted now anyway. thanks
Hi, I am not using the frigate addon in ha, i am using frigate in a container on proxmox but the blueprint wants a frigate camera. How do I go about this?
Hi there I have not setup frigate on a separate machine before so I do not have a direct means of testing your issues. I can only assume you need the frigate proxy add on or port 5000 needs opened on frigate server side.
Regarding the zone_names error… Did you select a zone from the predefined list or did you manually enter it.
Hi, i am trying to setup the automation and it triggeres. But never got the notification to fire. it starts a loop and i think when it checks if the clip exist, it does not. is ther something i am missing? I’m not sure about my downloader configuration… is there a way to access the files directly on my frigate box?