what error are you getting
Frigate LLM Notification v0.6 - Intelligent Security Camera Automation
An advanced Home Assistant automation blueprint that combines Frigate NVR, LLM Vision AI, and interactive mobile notifications to create intelligent, context-aware security alerts with actionable buttons.
IMPORTANT: Migration Notice for v0.6
Breaking Changes - Manual Migration Required
Version 0.6 introduces significant structural changes that prevent automatic migration from previous versions.
If you are upgrading from v0.43/0.5-dev or earlier:
Document your existing settings - Note down all your current automation configurations
Delete existing automations created with previous blueprint versions
Import the new v0.6 blueprint (replaces the old blueprint)
Create new automations using the v0.6 blueprint with your documented settings
Configure new button features and filtering options
Why manual migration is required:
- 18+ new button configuration inputs that didn’t exist in previous versions
- Enhanced filtering system with new person/zone validation
- Restructured template logic for button event handling
- New variable structure for safe event processing
Migration cannot be automated because:
- New inputs have no equivalent in old versions
- Template logic has been fundamentally restructured
- Button functionality requires explicit user configuration
- Filtering options need individual setup decisions
Key Features
Interactive Notifications
3 Customizable Buttons per notification type (New Event + End Event)
Smart Silence Feature with custom timer input (5-120 minutes)
Direct Call Action with configurable phone numbers
Notification Actions: View Live, View Snapshot, View Clip
Custom Actions: Execute any Home Assistant automation
Advanced AI Analysis
- Dual Analysis: Initial snapshot + comprehensive video clip analysis
- Multiple LLM SupportUsing LLM Vision: GPT-4, Claude, Gemini, Llama models
- Person-Verified Detection: Handles Frigate’s enhanced person detection
- Memory Integration: Uses LLM Vision timeline and memory features
- Custom Prompts: Tailored AI analysis for your specific needs
- Sub-label Support: Advanced filtering with Frigate sub-labels
Smart Filtering System
- Zone Vacancy Detection: Only alert when specific people are away
- Person Presence Validation: Check if family members are home/away
- Custom Conditions: Add complex logic conditions
Features
- Optional Downloader: Choose between direct URLs or local file storage
- Parallel Processing: Handle multiple camera events simultaneously
- Enhanced Error Handling: Robust retry logic and safe template processing
- Cooldown Management: Prevent notification spam with configurable delays
Prerequisites
Required Components
-
# Frigate 0.13+ with MQTT setup # Camera zones configured # Object detection enabled -
# Provider configured (OpenAI, Anthropic, Google, etc.) # API keys properly set # Memory/Timeline optional but recommended -
# Android/iOS app installed # Notification permissions enabled # Actionable notifications supported -
Downloader Integration (Optional)
# Built-in HA integration # Directory permissions configured # Storage space available
Installation & Migration
New Installation (v0.6)
1. Import Blueprint
# Method 1: Direct URL Import
https://raw.githubusercontent.com/willhaggan/HA_Frigate_VLLM_Notification/main/Latest.yaml
# Method 2: HA UI
Settings → Automations & Scenes → Blueprints → Import Blueprint
2. Basic Configuration
# Minimal setup for testing
Camera: camera.front_door
Objects: ["person"]
LLM Provider: your_llm_vision_provider
Notification Devices: [your_mobile_device]
Button 1: "View Live" (VIEW_LIVE)
Button 2: "Silence" (SILENCE)
3. Test Your Setup
- Create automation with minimal settings above
- Trigger person detection at your camera
- Verify notification appears with working buttons
- Test silence functionality with custom timer
Migration from Previous Versions
Step 1: Document Current Settings
Before making any changes, document your existing automations:
# For each existing automation, note:
Camera: camera.your_camera
Objects: ["person", "car"]
Zones: ["front_porch", "driveway"]
AI Prompts: "Your custom prompts"
Notification Devices: [your_devices]
Custom Actions: [any_custom_actions]
Downloader Settings: enabled/disabled
LLM Provider: your_provider
LLM Model: your_model
Step 2: Remove Old Automations
- Go to Settings → Automations & Scenes
- Find automations created with previous blueprint versions
- Document their settings (see Step 1)
- Delete the old automations
Step 3: Import New Blueprint
- Import v0.6 blueprint using the URL above
- Old blueprint will be replaced automatically
Step 4: Recreate Automations
- Create new automation from v0.6 blueprint
- Apply your documented settings
- Configure new button features:
# New Event Buttons Button 1: "View Live" → VIEW_LIVE Button 2: "Silence" → SILENCE Button 3: "Call" → CALL (optional) # End Event Buttons Button 1: "View Clip" → VIEW_CLIP Button 2: "View Live" → VIEW_LIVE Button 3: "Silence" → SILENCE - Set up new filtering options (optional):
# Advanced Filtering (Optional) Vacant Zone: zone.home Persons Away: ["person.john", "person.jane"] Custom Condition: "{{ now().hour >= 22 }}" # Example
Step 5: Test Everything
- Test each recreated automation
- Verify notifications work with new buttons
- Test silence functionality
- Confirm AI analysis still works as expected
Configuration Examples
Example 1: Basic Front Door Monitoring
# Simple person detection with interactive buttons
Frigate Camera: camera.front_door
Required Objects: ["person"]
Zones: ["front_porch"]
Severity: ["alert", "detection"]
# Interactive Buttons (New Event)
Button 1: "View Live" → VIEW_LIVE
Button 2: "Silence" → SILENCE
Button 3: "Call Security" → CALL
# AI Analysis
Snapshot Prompt: "Describe who is at the front door and what they appear to be doing"
Clip Prompt: "Analyze the person's behavior. Are they delivering something, visiting, or suspicious?"
Example 2: Advanced Driveway Security
# Vehicle detection with person vacancy checking
Frigate Camera: camera.driveway
Required Objects: ["car", "person"] # Requires BOTH objects
Zones: ["driveway"]
# Smart Filtering
Vacant Zone: zone.home
Persons Away: ["person.john", "person.jane"] # Only alert when family is away
Custom Condition: "{{ now().hour >= 22 or now().hour <= 6 }}" # Night only
# AI Analysis
Snapshot Prompt: "Vehicle and person detected in driveway while family is away. Describe the activity"
Clip Prompt: "Analyze this security event. Is this suspicious activity or legitimate?"
Example 3: Package Delivery Detection
# Smart package monitoring
Frigate Camera: camera.front_door
Required Objects: ["person"]
Zones: ["front_porch", "steps"]
Sub Labels: ["delivery", "package"]
# Notification Buttons
Button 1: "View Snapshot" → VIEW_SNAPSHOT
Button 2: "View Full Clip" → VIEW_CLIP
Button 3: "Silence 30min" → SILENCE
# AI Prompts
Snapshot Prompt: "Focus on package delivery activity. Describe what the person is doing with any packages"
Clip Prompt: "Analyze the complete delivery process. Was a package delivered or picked up?"
Interactive Button Features
Available Button Actions
| Action | Description | Use Case |
|---|---|---|
VIEW_LIVE |
Open live camera feed | Quick security check |
VIEW_SNAPSHOT |
View event snapshot | See what triggered alert |
VIEW_CLIP |
View full video clip | Review complete event |
SILENCE |
Pause notifications | Temporary quiet period |
CALL |
Dial phone number | Contact security/family |
Smart Silence Feature
# User interaction examples:
Tap "Silence" → 5 minutes (default)
Type "30" + Send → 30 minutes
Type "120" + Send → 2 hours
# System behavior:
- Automation temporarily disabled
- Visual confirmation shown
- Automatic re-enable after timer
- Affects only that specific automation
Button Configuration
# New Event Buttons
Button 1: "View Live" → VIEW_LIVE → Opens camera entity
Button 2: "Silence" → SILENCE → Enables text input for timer
Button 3: "Call" → CALL → Dials configured number
# End Event Buttons
Button 1: "View Clip" → VIEW_CLIP → Opens video file
Button 2: "View Live" → VIEW_LIVE → Opens camera entity
Button 3: "Silence" → SILENCE → Enables text input for timer
Advanced Template Variables
Use these in custom messages and AI prompts:
Event Information
{{input_objects}} # Objects required: ["person", "car"]
{{objects}} # Objects detected: ["person"]
{{camera_name}} # Camera name: "Front Door"
{{zone_names}} # Zones required: ["front_porch"]
{{before_zones}} # Initial zones: ["driveway"]
{{after_zones}} # Final zones: ["front_porch"]
{{detections[0]}} # Event ID: "1642095978.456789-abc123"
{{review_id}} # Review ID for clips
Custom Logic Examples
# Time-based conditions
{{ now().hour >= 22 or now().hour <= 6 }} # Night only
{{ now().weekday() < 5 }} # Weekdays only
# Person presence
{{ is_state('person.john', 'home') }} # John is home
{{ states('zone.home') | int > 0 }} # Anyone home
# Weather integration
{{ states('weather.home') == 'rainy' }} # Weather conditions
Performance Tips
AI Model Selection
# Fast models (real-time alerts):
- gpt-4o-mini
- gemini-2.0-flash-lite
- claude-3-5-haiku-latest
# Quality models (detailed analysis):
- gpt-4o
- claude-3-5-sonnet-latest
- gemini-2.0-flash
Token Optimization
# Snapshot Analysis: 50-100 tokens (quick description)
# Clip Analysis: 100-200 tokens (detailed analysis)
# Balance: Speed vs Detail based on your needs
Storage Management
# Optional Downloader setup:
Root Directory: "/config/www/frigate"
Sub Directory: "{{now().strftime('%Y-%m-%d')}}" # Daily folders
Cleanup: Implement periodic deletion of old files
Contributing
We welcome contributions! Here’s how to help:
Bug Reports
- Use GitHub Issues with detailed error logs
- Include Home Assistant and integration versions
- Provide configuration examples that reproduce the issue
Feature Requests
- Describe the use case and expected behavior
- Consider how it integrates with existing features
- Provide examples of how you’d configure it
Pull Requests
- Fork the repository and create a feature branch
- Test thoroughly with multiple scenarios
- Update documentation for any new features
- Follow existing code style and patterns
Acknowledgments
- Frigate - Outstanding NVR software
- LLM Vision - Excellent AI vision integration
- Home Assistant Community - Continuous support and feedback
- Beta Testers - Community members who helped refine v0.6 features
Version: v0.6
Author: whag
Last Updated: August 2025
Compatibility: Home Assistant 2024.1+, Frigate 0.13+
Quick Links
- Blueprint File - Ready to import
- GitHub Issues - Bug reports and feature requests
- Home Assistant Community - General discussion
- Frigate Documentation - Camera setup guide
- LLM Vision Setup - AI provider configuration
1 Like
Hi Whag,
I just imported your blueprint this morning and integrated into my local environment. Works perfectly with both Google Gemini and local Ollama.
I would like to have a feature request. as my home assistan is lan only and can only be accessible via wireguard, I’m receiving mobile push messages through a Nextcloud talk message via HA nextcloud integration. Is it possible to publish the AI generated {{msg}} in the variable section to be able to use by Custom Actions in you blueprint?
Thank you
Hi there, Can you try this variable
"{{response.response.text}}"
Hi Whag, sorry for the delay in responding I’ve been away.
So I can’t get v5 to add for some reason so I’m working on v4.3 I’ve done some debugging and I’ve found I had several issues but I believe I’ve rectified most but I can’t seem to get the automation to fully work probably some amateur mistake I’m making but it’s failing at the download part.
So I have tested the downloader in the developer section and it downloads files correctly but when the automation runs I don’t see any new files appear in the folder my downloader folder is homeassistant/llm_downloads/ but when I tested it in developer mode I needed to use just /llm_downloads/ I’ve tried both in the automation but neither seem to work. Maybe I’m not leaving the time set long enough.
Also does the download get deleted after some time?
Hi there, can you remove any older versions and install the latest v0.6. after you have that installed it let me know if you have any issues. The downloader integration is totally optional and if enabled it needs maintained separately.
The downloader folder is your case should just be entered
llm_downloads
If you check the trace scroll down to check what value is assigned to the downloader variable it will tell you what is going wrong.
I would first install the latest version, and then redo the automations as they are not compatible with 0.6.
I removed the v0.43 this morning and added v0.6 and just did the setup as basic and it worked fine I’m so pleased I’ve spent days working through what was wrong.
I will try some more advanced features next.
What is the bonus to using the downloader please?
That’s good to hear, the downloader integration downloads the frigate images and clips to the HA machine before passing it to llm vision for analysis, the integration has an event trigger on successful file downloading so we know for certain the file will be there before running the analyse actions. It obviously has some cons mostly being the requirement for self maintaining although very simple to achieve with a shell command and the fact you can have multiple copys of the same image or clip on your local storage depending on your frigate and llm vision setups.
1 Like
Thanks, I’m now back from my trip away and I’m loving this automation but how do I set it to not repeat notifying me of the same object so basically when my car is on the drive it will continue periodically to send a notification about my car being on the drive
Hi Whag, sorry for the delayed answer, I was on vacation :). So no unfortunately the “{{response.response.text}}” variant is not working home assistant can not run the automation with:
“error”: “UndefinedError: ‘response’ is undefined”,
@Whag Amazing work.
Would it please be possible to add some customisation options for the image/video displayed in the notification itself (not triggered by the buttons)?
For example, the standard frigate notification blueprint enables you to select
thumbnail, snapshot, review gif, and object gifts for the initial and end notifications, along with video clips or live video.
Particularly I’d be happy with a snapshot followed by an m3u8 video clip (for IOS) as soon as that is available.
For the standard frigate notification blueprint, this format seems to link to a m3u8 video link that works in the browser, though I’ve yet to manage to make it play inside the notification.
{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/master.m3u8
where for my example
{{base_url}} = nabu casa url ending in .casa
{{id}} is extracted from mqtt
{{camera}} is the frigate camera name
Thanks for your consideration
hi there, you should acheive this with this type of zoning see the frigate examples below
https://docs.frigate.video/configuration/zones/
In frigate you could try somehting like this, create a ‘drive_entry’ zone and a ‘drive’ zone and filter alerts in ‘drive_entry’ to car only and then in the blueprint require drive_entry and drive in your zones.
no worries thanks for the details i will create a new variable to hold the response in the update
Okay so you need the m3u8 made available, i can certainly do this although i will not have a device to test it on but you can let me know, i will add this to the next update
Hi there there is limitations what can be shown in the notification see the link
So what happens when you click on the image/ clip or try to view the clip via a button on IOS, i ask because i do not have one to test, Also does the GIF play on the end event notification?
Hi just looking at this again the variable i asked you to test was wrong, it should be ‘response.response_text’ not ‘response.response.text’ my mistake, i will create 2 new variables anyway incase this does not work. One for the new event and 1 for the end event
I will add offline and test.
Thanks
Thanks very much. Apologies for the delay, have been travelling with limited access.
So I can confirm that with the standard Frigate notification blueprint gifs are updated.
I will check again tomorrow if this is the case for your blueprint when I take control and add the gif link instead of live video in the notification.
For m3u8… this seems to open in the browser on a button link press, but generates an error code when placed in the notification 
so maybe a no go for now (for me at least).
Hi there this blueprint is using the gif notification link for the end review event and the snapshot for the new event. That is fixed code for now so you should already have this in place
Since a couple days ago my notifications have started acting odd I no longer recurve them in real time but on a morning a seem to get a load from the previous day as though they just happened not sure what’s changed
Hi,
So the gif notification link works, but as you said… it’s a link rather than showing the gif in the notification body. I can edit that myself by taking control, but it would be nice to have the notification image/gif/video as a selectable option for the blueprint, like you provide for the links/click action.
I’ll also let you know if I have any progress on the iOS video. For now it is only viewable in the browser for me.
Thanks for all