Im trying to automate my backyard lights to turn off if no motion is detected. Currently I am using Reolink cameras that have a motion detected sensor, however I cant reasonably rely on that as im finding its not accurate and flags motion even if there is no animals or humans in the backyard.
I would like to use LLM Vision to analyze the feed(or snapshot) from my reolink camera and have LLM Vision tell me whether or not there are dogs or humans in the backyard in the feed, and if no then use that as help for a trigger to off the backyard lights, and if yes, then dont turn off the lights.
I dont know if this is possible with LLM vision currently. Im also open to other ideas on how people are automating turning off their backyard lights if there is nobody there
I’ve tried the exact same here and for the same reason. Also with a reolink camera. I have setup a connection to use Gemini. It can often tell me what it sees, how many persons, how many trees and such. Whatever i ask for. But it needs a trigger to start with. I use the motion trigger of the camera and also from available motion detectors. It work for most of the time but I do not like to throw those camera images online. Now i am investigating the use lof a local llm/vlm. I got it working with LM Studio but haven’t yet the time to test it thouroghly.
General flow is something triggers. That fires some kind of assessment of camera image, then returns (there’s plenty of examples of this won’t rehash)
But then to do what you want you will need to capture that in a response variable. Then pass the variable to an ai conversation with your prompt and the response as additional context.
TL;dr
Basically yes it’s possible but ai_task can’t act, you need an assist enabled conversation agent. So trigger > ai_task output > conversation
You might need to take over here with your own AI for various reasons, but in the Reolink App you can set these cameras up to sensitivity levels on people, vehicles, animals, all depending on the camera. I just did this this morning. Make sure the camera firmware is up to date. Reolink is tweaking the internal AI stuff constantly and it’s getting better and better.