Detecting snow on a pixel basis

Configuration
1

I need to detect snow and my plan is to use a webcam:

step 1
Crop the relevant image are (in my case, where grass/snow is)

step 2
check all pixels if the blue channel is larger than the red channel and the values are similar (aka white).

step 3
If number is higher than x set sensor to on (snow)

It only has to work at daylight.

Is this possible within Home Assistant?

2

Did you devise a system to detect snowfall?

3

Kind of, but it was too unreliable.

I will probably try again with ai to detect sunshine or snow using a pretrained model.

4

Just use an ai model. If I can use Gemini to figure out whether or not my trash can is out by the street by looking at a camera image, surely it could recognize snow in a picture.

2 Likes
5

I have to try that.

Is it really as simple as prompting is it sunny and it will set a sensor true or false?

And if I do this below the threshold of 1500 prompts a day it is even free?

6

ooh wow I didnt even think of this, I have Ollama running to provide descriptions into my Frigate NVR alerts … I could setup something to query if its snowing and turn on my heated mats!

Thank you for the idea :slight_smile: love it!

7

Would you mind to go into detail about that. I tried Gemini, but it isn’t available for me. How is the setup?

8

Here is the documentation for GenAI in Frigate:

Here is the Ollama setup:

but basically, you would setup Ollama on a host machine with your GPU, and I believe it automatically sets up as a server, you can then modify the Frigate config files to use Ollama.

Now, trying to get it to recognize snow will be something I have to look into, not sure if I can do it in Frigate or maybe I’ll do it in Home Assistant.

9

Thanks, I managed to test some models in olllama but the accuracy is not where I need it to be. the same image gives me mixed results. Might need to wait for better models.

10

what prompt did you ask?

I’d probably think saying something along the lines:

  • anaylze the footage from my outdoor camera, can you tell if it is currently snowing and not that there is snow on the ground, answer yes or no with 100% certainty.