here is what you can do
1 Like
Where in the blueprint can i add the code ?
I tried do it like you are ,but cant figure out what i need to do in the notification input (the last part in your screenshot )
Found the problem I had around “old image” being included in the notifications. I suspected it to be a cache problem, and indeed it is. I am using Cloudflare for my domain and Cloudflared tunnel and it turns out Cloudflare was caching the image and because the names of the images are the same (e.g. _0.jpg) I always got a cached (old) version of the image.
I solved it by introducing a cache rule to not cache “jpg” images. As described here: Home Assistant sends cached images in iOS notification? - #8 by msl2424
2 Likes
I am looking for a way to generate PNG images to display them in my display which is in LVGL esphome and which only takes transparent PNG, do you have any solutions please
I don’t use blueprint but a new automation with LLM vision
in LLM vision is it possible to add Deepseek
FYI before anyone decides to spend $ on the OpenAI api, I would suggest you try Gemini first. I’ve ran into multiple prompts where OpenAI just doesn’t like the prompt I’m feeding it and says “sorry I can’t help with that”. Gemini seems to be a lot more straight forward with the information it spits out where OpenAI tries to go for a human touch which isn’t what I’m trying to accomplish with the security prompts.
I’ll post my node-red example flow here soon, I’m sure someone can improve on what I built.
Node Red Example Flow here.
Few notes, I’m using the following integrations to help:
- Google Cast, specifically Google Home Hubs (Can be switched with Alexa or whatever you have)
- Frigate
- Unifi Protect
- Companion App.
Frigate might be unnecessary in this case, a binary sensor can be used to replace. Same thing with Unifi Protect, just change to your own camera setup. Something however has to trigger the person detection.
I also removed my provider from these flows because it feels like it could be tied to my API key, you can find your provider by going to Developer Tools → going to actions → selecting “LLM Vision. Image Analyzer”, and at the bottom clicking on “go to yaml mode”.
Paste the provider and include quotes
[{"id":"b2b424ddbd92c0e2","type":"tab","label":"Example Security Flow","disabled":false,"info":"","env":[]},{"id":"2cdded6300218d5e","type":"comment","z":"b2b424ddbd92c0e2","name":"Driveway AI Warning, CLICK ON COMMENT NODES FOR FURTHER EXPLAINATION","info":"","x":340,"y":40,"wires":[]},{"id":"662d722c631b6961","type":"server-state-changed","z":"b2b424ddbd92c0e2","name":"Occupancy detected for 500 MS","server":"2e31528c.f3973e","version":6,"outputs":2,"exposeAsEntityConfig":"","entities":{"entity":["binary_sensor.driveway_person_occupancy"],"substring":[],"regex":[]},"outputInitially":false,"stateType":"str","ifState":"on","ifStateType":"str","ifStateOperator":"is","outputOnlyOnStateChange":true,"for":".5","forType":"num","forUnits":"seconds","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"eventData"},{"property":"topic","propertyType":"msg","value":"","valueType":"triggerId"}],"x":130,"y":280,"wires":[["79b5817c845a0263"],[]]},{"id":"911940e0ea4500d5","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Driveway camera snapshot","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"camera.snapshot","floorId":[],"areaId":[],"deviceId":[],"entityId":["camera.driveway_high"],"labelId":[],"data":"{\"filename\":\"/media/snapshots/driveway_person.jpg\"}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"camera","service":"snapshot","x":960,"y":280,"wires":[["27de42ed98e84e6f"]]},{"id":"0bb2f0386aa72212","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Ask Google about person","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"llmvision.image_analyzer","floorId":[],"areaId":[],"deviceId":[],"entityId":[],"labelId":[],"data":"{\t \"provider\":\"xxx\",\t \"model\":\"gemini-1.5-pro\",\t \"message\":\"Scare the person by describing them, they are most likely attempting to break in, keep it short.\",\t \"image_file\":\"/media/snapshots/driveway_person.jpg\",\t \"target_width\":650,\t \"max_tokens\":100,\t \"temperature\":0.3,\t \"include_filename\": \"true\"\t}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[{"property":"response_text","propertyType":"msg","value":"","valueType":"results"}],"queue":"none","blockInputOverrides":false,"domain":"llmvision","service":"image_analyzer","x":1890,"y":260,"wires":[["767e40a9c729037c"]]},{"id":"27de42ed98e84e6f","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Send to GoogleAI","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"llmvision.image_analyzer","floorId":[],"areaId":[],"deviceId":[],"entityId":[],"labelId":[],"data":"{\t \"provider\":\"xxx\",\t \"model\":\"gemini-2.0-flash-exp\",\t \"message\":\"Reply with a simple yes or no; is person in the photo holding a package or is there a UPS, USPS, FedEx, or Amazon truck in the picture?\",\t \"image_file\":\"/media/snapshots/driveway_person.jpg\",\t \"target_width\":650,\t \"max_tokens\":100,\t \"temperature\":0.3,\t \"include_filename\": \"true\"\t}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[{"property":"response_text","propertyType":"msg","value":"","valueType":"results"}],"queue":"none","blockInputOverrides":false,"domain":"llmvision","service":"image_analyzer","x":1190,"y":280,"wires":[["3c90dbfb4c71fc25","b38f7bf29cfdce23"]]},{"id":"b38f7bf29cfdce23","type":"switch","z":"b2b424ddbd92c0e2","name":"Check for yes/no","property":"response_text.response_text","propertyType":"msg","rules":[{"t":"cont","v":"Yes","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1430,"y":280,"wires":[["be94bd0220304c46"],["0bb2f0386aa72212","695bfa6b95cdfefe","f4abfbf69ddc08f2"]]},{"id":"422b9ae59a7b796d","type":"change","z":"b2b424ddbd92c0e2","name":"","rules":[{"t":"set","p":"payload","pt":"flow","to":"response_text.response_text","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":2320,"y":260,"wires":[["edabed6540fa80fa","9edbb302b654e93d"]]},{"id":"767e40a9c729037c","type":"function","z":"b2b424ddbd92c0e2","name":"Trim Newline","func":"// Strip the newline character from msg.response_text\nif (msg.response_text.response_text) {\n msg.response_text.response_text = msg.response_text.response_text.trim();\n}\nreturn msg;","outputs":1,"timeout":"","noerr":0,"initialize":"","finalize":"","libs":[],"x":2130,"y":260,"wires":[["422b9ae59a7b796d"]]},{"id":"695bfa6b95cdfefe","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Set the Driveway speaker to 100%","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"media_player.volume_set","floorId":[],"areaId":[],"deviceId":[],"entityId":["media_player.driveway_speaker"],"labelId":[],"data":"{\"volume_level\":1}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":true,"domain":"media_player","service":"volume_set","x":1720,"y":340,"wires":[[]]},{"id":"20a4e389b36e32b3","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"TTS Driveway","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"tts.speak","floorId":[],"areaId":[],"deviceId":[],"entityId":["tts.home_assistant_cloud"],"labelId":[],"data":"{\"message\":\"{{flow.payload}}\",\"media_player_entity_id\":\"media_player.driveway_speaker\",\"cache\":\"true\"}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"tts","service":"speak","x":3400,"y":260,"wires":[[]]},{"id":"edabed6540fa80fa","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Phone Notification - Dan","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"notify.mobile_app_dan_pixel_8_pro","floorId":[],"areaId":[],"deviceId":[],"entityId":[],"labelId":[],"data":"{\"title\":\"Warning sent to intruder\",\"message\":\"Message: {{flow.payload}}\",\"data\":{\"channel\":\"Intruder\",\"tag\":\"intruder-driveway\",\"image\":\"/media/local/snapshots/driveway_person.jpg\",\"clickAction\":\"/lovelace/cameras\",\"priority\":\"high\",\"ttl\":0}}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"notify","service":"mobile_app_dan_pixel_8_pro","x":2570,"y":200,"wires":[[]]},{"id":"79b5817c845a0263","type":"api-current-state","z":"b2b424ddbd92c0e2","name":"Package Expected","server":"2e31528c.f3973e","version":3,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","entity_id":"input_boolean.test_driveway_warning","state_type":"str","blockInputOverrides":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":470,"y":280,"wires":[["01a241bb67ffbc3f"],[]]},{"id":"057d5a368b0e4a91","type":"inject","z":"b2b424ddbd92c0e2","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":1640,"y":180,"wires":[["0bb2f0386aa72212"]]},{"id":"1b7b618912b6b64e","type":"inject","z":"b2b424ddbd92c0e2","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":1020,"y":200,"wires":[["27de42ed98e84e6f"]]},{"id":"3c90dbfb4c71fc25","type":"debug","z":"b2b424ddbd92c0e2","name":"debug 3","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1360,"y":500,"wires":[]},{"id":"9edbb302b654e93d","type":"api-current-state","z":"b2b424ddbd92c0e2","name":"Driveway warning test is on","server":"2e31528c.f3973e","version":3,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","entity_id":"input_boolean.test_driveway_warning","state_type":"str","blockInputOverrides":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":2580,"y":260,"wires":[[],["e8e6da043d261007"]]},{"id":"01a241bb67ffbc3f","type":"delay","z":"b2b424ddbd92c0e2","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"5","rateUnits":"minute","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"allowrate":false,"outputs":1,"x":680,"y":280,"wires":[["911940e0ea4500d5"]]},{"id":"e8e6da043d261007","type":"time-range-switch","z":"b2b424ddbd92c0e2","name":"","lat":"35.69052","lon":"-78.29317","startTime":"23:00","endTime":"07:00","startOffset":0,"endOffset":0,"x":2970,"y":260,"wires":[["2f23a242efe854c2"],[]]},{"id":"d62465aac319339d","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Set Volume Guest Room Display","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"media_player.volume_set","floorId":[],"areaId":[],"deviceId":[],"entityId":["media_player.guest_room_display"],"labelId":[],"data":"{\"volume_level\":0.7}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"media_player","service":"volume_set","output_location":"","output_location_type":"none","x":2454,"y":442,"wires":[["053bc8be21fc1eee"]]},{"id":"c96cd6bb8b93554a","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"TTS Guest Room","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"tts.speak","floorId":[],"areaId":[],"deviceId":[],"entityId":["tts.home_assistant_cloud"],"labelId":[],"data":"{\t \"message\":\"Person Spotted in the driveway, please check\",\t \"media_player_entity_id\": \"media_player.guest_room_display\",\t \"cache\":\"true\"\t}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"tts","service":"speak","x":2944,"y":442,"wires":[["9c5e2c2b06905886"]]},{"id":"053bc8be21fc1eee","type":"delay","z":"b2b424ddbd92c0e2","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":2700,"y":440,"wires":[["c96cd6bb8b93554a"]]},{"id":"9c5e2c2b06905886","type":"delay","z":"b2b424ddbd92c0e2","name":"","pauseType":"delay","timeout":"4","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":3134,"y":442,"wires":[["afe5b1fa2b6cf289"]]},{"id":"afe5b1fa2b6cf289","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"TTS Guest Room","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"tts.speak","floorId":[],"areaId":[],"deviceId":[],"entityId":["tts.home_assistant_cloud"],"labelId":[],"data":"{\t \"message\":\"Person Spotted in the driveway, please check\",\t \"media_player_entity_id\": \"media_player.guest_room_display\",\t \"cache\":\"true\"\t}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"tts","service":"speak","x":3344,"y":442,"wires":[["dfd1286d79b8553d"]]},{"id":"5d036f540a7cf592","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Media Stop","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"media_player.turn_off","floorId":[],"areaId":[],"deviceId":[],"entityId":["media_player.guest_room_display","media_player.master_bedroom_display","media_player.downstairs_office_display","media_player.upstairs_display"],"labelId":[],"data":"","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"media_player","service":"turn_off","output_location":"","output_location_type":"none","x":4104,"y":442,"wires":[[]]},{"id":"dfd1286d79b8553d","type":"delay","z":"b2b424ddbd92c0e2","name":"","pauseType":"delay","timeout":"4","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":3554,"y":442,"wires":[["7b62a977eca43fe1"]]},{"id":"7b62a977eca43fe1","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Stream Camera","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"camera.play_stream","floorId":[],"areaId":[],"deviceId":[],"entityId":["camera.driveway_medium_resolution_channel"],"labelId":[],"data":"{\t \"media_player\": [\t \"media_player.downstairs_office_display\",\t \"media_player.master_bedroom_display\",\t \"media_player.upstairs_display\",\t \"media_player.guest_room_display\"\t ]\t}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"camera","service":"play_stream","output_location":"","output_location_type":"none","x":3734,"y":442,"wires":[["34fc28d0d7c302c1"]]},{"id":"34fc28d0d7c302c1","type":"delay","z":"b2b424ddbd92c0e2","name":"","pauseType":"delay","timeout":"6","timeoutUnits":"minutes","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":3914,"y":442,"wires":[["5d036f540a7cf592"]]},{"id":"f4abfbf69ddc08f2","type":"api-current-state","z":"b2b424ddbd92c0e2","name":"Driveway warning test is on","server":"2e31528c.f3973e","version":3,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","entity_id":"input_boolean.test_driveway_warning","state_type":"str","blockInputOverrides":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":1700,"y":460,"wires":[[],["14d8d3497464e5cb"]]},{"id":"14d8d3497464e5cb","type":"time-range-switch","z":"b2b424ddbd92c0e2","name":"","lat":"35.69052","lon":"-78.29317","startTime":"23:00","endTime":"07:30","startOffset":0,"endOffset":0,"x":1964,"y":462,"wires":[["836795b1e2561ab6"],[]]},{"id":"87d8bc9478388799","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Turn on Guest Lights","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"light.turn_on","floorId":[],"areaId":[],"deviceId":[],"entityId":["light.guest_room_fan_light","light.guest_room_light"],"labelId":[],"data":"","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"light","service":"turn_on","x":2414,"y":502,"wires":[[]]},{"id":"be94bd0220304c46","type":"api-call-service","z":"b2b424ddbd92c0e2","name":"Phone Notification - Dan","server":"2e31528c.f3973e","version":7,"debugenabled":false,"action":"notify.mobile_app_dan_pixel_8_pro","floorId":[],"areaId":[],"deviceId":[],"entityId":[],"labelId":[],"data":"{\"title\":\"Delivery Person Spotted\",\"message\":\"Delivery person spotted, please check\",\"data\":{\"channel\":\"Delivery Driver\",\"tag\":\"delivery-driveway\",\"image\":\"/media/local/snapshots/driveway_person.jpg\",\"clickAction\":\"/lovelace/cameras\"}}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","blockInputOverrides":false,"domain":"notify","service":"mobile_app_dan_pixel_8_pro","x":1690,"y":100,"wires":[[]]},{"id":"836795b1e2561ab6","type":"api-current-state","z":"b2b424ddbd92c0e2","name":"Front Door Locked","server":"2e31528c.f3973e","version":3,"outputs":2,"halt_if":"locked","halt_if_type":"str","halt_if_compare":"is","entity_id":"lock.front_door","state_type":"str","blockInputOverrides":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":2164,"y":462,"wires":[[],["d62465aac319339d","87d8bc9478388799"]]},{"id":"94f997700e010570","type":"inject","z":"b2b424ddbd92c0e2","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":3754,"y":382,"wires":[["5d036f540a7cf592"]]},{"id":"67cf298f9251ee69","type":"inject","z":"b2b424ddbd92c0e2","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":3320,"y":360,"wires":[["7b62a977eca43fe1"]]},{"id":"f3c69ea892f4a7a1","type":"comment","z":"b2b424ddbd92c0e2","name":"Delay Occupancy slightly for a better picture","info":"Using Frigate in this instance, I delay\nthe occupancy detection slightly to get \na better picture. Can change this with\nUnifi or another binary sensor such as \na motion sensor.","x":170,"y":220,"wires":[]},{"id":"b1ac5f51d35efd44","type":"comment","z":"b2b424ddbd92c0e2","name":"Check if Package Expected Boolean is True, Rate limit messages","info":"Created a manual boolean for Amazon flex\npackages, since there isn't a way to \npoll Amazon to see if a package is being\ndelievered overnight.\n\nRate limit messages to 1 every 5 minutes","x":610,"y":220,"wires":[]},{"id":"84f474b8634a9775","type":"comment","z":"b2b424ddbd92c0e2","name":"Take a picture, create an initial prompt","info":"Note: I removed my provider from this \nmessage, you can find your provider by\nopening Developer Tools, going to actions,\nselecting \"LLM Vision. Image Analyzer\", and \nat the bottom clicking on \"go to yaml mode\"\nPaste the provider and include quotes\n\nTake a snapshot using a camera integrated\nwith HA. Using that image, send it to\nGemini and ask it to identify delivery\ntrucks or potential packages to avoid\nsending unnecessary messages.\n\nUsing a switch node, check Gemini response\nand route as necessary.\n\nCan use gemini 1.5 flash here","x":990,"y":340,"wires":[]},{"id":"e171df5880556c4a","type":"comment","z":"b2b424ddbd92c0e2","name":"Send a package notification message","info":"Send a simple package notification\n message with image.","x":1710,"y":60,"wires":[]},{"id":"d6a498a79071a0d0","type":"comment","z":"b2b424ddbd92c0e2","name":"Take a picture, create an initial prompt","info":"Output debug message, should be under\ndata -> response text -> response text","x":1430,"y":540,"wires":[]},{"id":"1ad2c0b72b8c2b2d","type":"comment","z":"b2b424ddbd92c0e2","name":"Set Camera Volume","info":"Prep to speak out the camera","x":1670,"y":380,"wires":[]},{"id":"494b23d15d482e4b","type":"comment","z":"b2b424ddbd92c0e2","name":"Test mode Boolean","info":"Send if test mode is off","x":1670,"y":500,"wires":[]},{"id":"80b1f69e9f45a632","type":"comment","z":"b2b424ddbd92c0e2","name":"Ask about the person, trim the response for invalid charactes, set flow variable","info":"Note: I removed my provider from this \nmessage, you can find your provider by\nopening Developer Tools, going to actions,\nselecting \"LLM Vision. Image Analyzer\", and \nat the bottom clicking on \"go to yaml mode\"\nPaste the provider and include quotes\n\nPrompt to ask gemini about the person,\nonce response is received, trim the\nresponse (new line character caused error)\n\nSet a global variable because for some\nreason I can't make it happy\n","x":2050,"y":200,"wires":[]},{"id":"a10b33b4e299326e","type":"comment","z":"b2b424ddbd92c0e2","name":"Test mode Boolean","info":"Send if test mode is off","x":2550,"y":300,"wires":[]},{"id":"5795ab35434ccf44","type":"comment","z":"b2b424ddbd92c0e2","name":"Send a notification to phone with image and message","info":"Send a notification to phone with image\nand Gemini response","x":2660,"y":140,"wires":[]},{"id":"a855f29d0a923f68","type":"comment","z":"b2b424ddbd92c0e2","name":"Check if between time, TTS from Camera","info":"Check if between listed hours and TTS at \nperson on camera using response","x":3080,"y":180,"wires":[]},{"id":"2f23a242efe854c2","type":"api-current-state","z":"b2b424ddbd92c0e2","name":"Front Door Locked","server":"2e31528c.f3973e","version":3,"outputs":2,"halt_if":"locked","halt_if_type":"str","halt_if_compare":"is","entity_id":"lock.front_door","state_type":"str","blockInputOverrides":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":3190,"y":260,"wires":[[],["20a4e389b36e32b3"]]},{"id":"aea7763958c6a300","type":"comment","z":"b2b424ddbd92c0e2","name":"Turn on Lights in Room, Set Notification Volume, then stream camera to multiple google nest hubs for 6 minutes","info":"","x":2720,"y":380,"wires":[]},{"id":"5d41038ba4521936","type":"comment","z":"b2b424ddbd92c0e2","name":"Note if using unifi camera","info":"Unifi camera stream was too high for\ngoogle nest hubs and caused buffering.\n\nSuggest using Medium Resolution RSTP stream from\nUnifi instead of high resolution.","x":3750,"y":500,"wires":[]},{"id":"2e31528c.f3973e","type":"server","name":"Home Assistant","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":"30","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true}]
1 Like
Hi all
anyone using LLM Vision with frigate-notify ?
I am using frigate-notify to send a notification via Telegram with a snapshot and a link to local frigate event clip and also to send a webhook to do some other automation.
Is there a way to analyze with LLM vision a jpg or mp4 having only the link and not with a local filename?
Thanks
DEEPSEEK V3 LLM Vision is it possible in the future
Hi all
I’m trying to use llm vision in Nodered using the llmvision.video_analyzer action
The config node is this
and the json data are these
{ "provider": "01JGVZYCCTSVSHKWD5W0701P7B",
"model": "gemini-1.5-flash",
"message": "Descrivi cosa vedi, solo se sono persone.",
"remember": true,
"video_file": "",
"event_id": msg.payload.id,
"max_frames": 5,
"target_width": 1280,
"detail": "high",
"max_tokens": 100,
"temperature": 0.2
}
I’m using msg.payload.id as Frigate event_id
But I’m getting error reporting invalid json
and in the node json visual editor I get this
But I don’t see the error.
Can you help, please?
Thanks
Hi
Trying to install it and getting
Could not connect to the server. Check you API key or IP and port
when adding my Open Ai api key .
any ideas ?
Thanks for the Blueprints. Needing assistance with one part. For the notification on my phone, I am having trouble getting it to open to the image (snapshot). Is {{video if
video != ‘’‘’ else image}} the correct entry to use for that to happen?
Hi, i just discover this amazing blueprint and integration. I setup LocalAI and llm integration and the blueprint. Im looking for documentation for the blueprint and LocalAI. Witch model i need to use in LocalAI ? and on the blueprint how i can set the path of my frigate and i how can tell localai to tell me a description ?
Thank you for putting this together! It is extremely helpful (and fun) to use. I’ve set everything up to use Ollama and Llava and things are working great. I got the Events Calendar working as well. Now, i’m trying to figure out how to get things set up with Assist so that I can ask it about remembered events, but I can’t figure out how to get this working with Ollama. The documentation only points to how to get things working with Extended OpenAI Conversation, but no other examples. Any help is appreciated!
As a quick follow up question, i’d also love to know if there is a way to attach the image to the event in the event calendar so that it isn’t just the LLM response without the contextual image.