Query Ollama with images

I’d like to request a feature with the Ollama integration.

It would be very usefull to be able to send an image when asking something to a vision model in Ollama. This could unlock so many use cases because automations would be able to gather so many information using the surveillance cameras.

Some examples

  • It would be possible to see if a door or a window is open without extra sensors
  • It would be possible to detect people in a room and their activities to set scenes automatically
  • It would be possible to detect if an animal is present where it shouldn’t be

Some use cases are already possible with image recognition softwares but it is not very flexible while vision LLMs are less precise, they have so much more potential…