From this:
To this:
The process is the following:
- Take a sample of the water using the test strip;
- Take a photo of the strip and the reference values on the packing;
- Upload the photo to Home Assistant directly from the phone;
- The photo gets sent to an LLM engine for analysis;
- Parsing the results and feeding some sensors with it.
Integrations used:
- Folder watcher - fires up and event every time a new file appears in a specific folder. This is the trigger for the automation.
- LLM Vision - Sends the image to an LLM engine for analysis.
- MQTT - I find it easy to create a manual MQTT sensor in HA and the use MQTT Publish to feed it with data. The short text status is the sensor value. The actual measurements are fed into the sensor as attributes.
- Template sensors - To extract the attributes into separate sensors.
MQTT sensor definition:
mqtt:
sensor:
- name: "Параметри на водата"
unique_id: aquarium_water_parameters
state_topic: "homeassistant/sensor/aquarium/water/state"
json_attributes_topic: "homeassistant/sensor/aquarium/water/attributes"
The automation:
alias: Анализа на параметрите на водата (LLM)
description: ""
triggers:
- trigger: event.received
target:
device_id: 68e0feb5601fc143a309e991b8ed83de
options:
event_type:
- closed
enabled: false
- trigger: event
event_type: folder_watcher
event_data:
event_type: closed
conditions: []
actions:
- action: llmvision.image_analyzer
metadata: {}
data:
store_in_timeline: true
use_memory: false
include_filename: true
target_width: 1280
max_tokens: 500
generate_title: true
expose_images: false
response_format: json
title_field: title
description_field: description
provider: 01KCQCZEDGQCB9N3AGNJE26672
message: |2-
Aquarium water parameters analysis. The photo has the legend and the sample
strip. The water is generaly hard and alkaline. That is just the way it is.
The tank has 6 neon tetras, 3-5 cherry shrimps, some plants, snails and
driftwood. Respod as a JSON structured water parameter data. Include a text
field containing a narrative description of the water condition in Macedonian. Do not use
ranges in the response data. Make your best guess.
image_file: "{{ trigger.event.data.path }}"
structure: "{\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"Event title\"\n },\n \"text\": {\n \"type\": \"string\",\n \"description\": \"Water quality description\"\n },\n \"ph\": {\n \"type\": \"number\"\n },\n \"total_alkalinity_mg_l\": {\n \"type\": \"number\"\n },\n \"general_hardness_mg_l\": {\n \"type\": \"number\"\n },\n \"total_chlorine_mg_l\": {\n \"type\": \"number\"\n },\n \"free_chlorine_mg_l\": {\n \"type\": \"number\"\n },\n \"copper_mg_l\": {\n \"type\": \"number\"\n },\n \"iron_mg_l\": {\n \"type\": \"number\"\n },\n \"nitrate_mg_l\": {\n \"type\": \"number\"\n },\n \"nitrite_mg_l\": {\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"title\",\n \"text\",\n \"ph\",\n \"total_alkalinity_mg_l\",\n\t\"general_hardness_mg_l\",\n\t\"total_chlorine_mg_l\",\n \"free_chlorine_mg_l\",\n \"copper_mg_l\",\n \"iron_mg_l\",\n \"nitrate_mg_l\",\n \"nitrite_mg_l\"\n ],\n \"additionalProperties\": false\n}"
response_variable: response_data
- choose:
- conditions:
- condition: template
value_template: "{{ response_data.response_text is defined }}"
sequence:
- action: mqtt.publish
metadata: {}
data:
retain: true
topic: homeassistant/sensor/aquarium/water/state
payload: "{{ response_data.response_text }}"
qos: "0"
default:
- action: mqtt.publish
metadata: {}
data:
retain: true
topic: homeassistant/sensor/aquarium/water/state
payload: "{{ response_data.structured_response.text }}"
qos: "0"
- action: mqtt.publish
metadata: {}
data:
retain: true
topic: homeassistant/sensor/aquarium/water/attributes
payload: |-
{
"Text": "{{ response_data.structured_response.text }}",
"ph": "{{ response_data.structured_response.ph }}",
"total_alkalinity_mg_l": "{{ response_data.structured_response.total_alkalinity_mg_l }}",
"general_hardness_mg_l": "{{ response_data.structured_response.general_hardness_mg_l }}",
"total_chlorine_mg_l": "{{ response_data.structured_response.total_chlorine_mg_l }}",
"free_chlorine_mg_l": "{{ response_data.structured_response.free_chlorine_mg_l }}",
"copper_mg_l": "{{ response_data.structured_response.copper_mg_l }}",
"iron_mg_l": "{{ response_data.structured_response.iron_mg_l }}",
"nitrate_mg_l": "{{ response_data.structured_response.nitrate_mg_l }}",
"nitrite_mg_l": "{{ response_data.structured_response.nitrite_mg_l }}"
}
qos: "0"
enabled: true
mode: queued
max: 10
Notes:
- The folder watcher fires up several event on file upload. That last one in the sequence has
event_type: closed. This is the one we want. - There are two trigger clauses in the automation, but only one is enabled. Started this project using
event.receivedbut this ended being unreliable. Events would fire, but the automation wouldn’t tigger. I moved to oldereventtype trigger and everything works now. I might need some work in the future is i add more folder watchers. - There is a condition in the automation that checks for existence of
response_data.response_textin the result. This means thatllmvision.image_analyzerreturned an error. This error is then populated into the sensor as value. Otherwise, parse the result and populate the sensor and the attributes with the response data.