SDI-12 Logger through HomeAssistant

Hi

I’m using a Python script which I found in the net and adapted it to my needs. The script is started by a cron job and writes all data (not only the EC,WC and temp.) to a Maria DB. In addition I get some weather data from a weather station to the DB. With NVD3 I have created some graphs to be presented in a browser to show the dependencies. Pumps and lights are connected with an Gembird EG-PMS2 USB and can be switched on/off by command line and cron job.
BR,
Michael

1 Like

Thanks! Can I ask what formula you used for converting your raw EC values?

Hi, can you please specify how you get it displayed on grafana? Did you succeed eventually?

Hi Rik,
I tried to copied what you had done in this topic. I'm running HA on raspberry pi 4b and using the Liudr usb adaptor to read Acclima [TDR315H](https://acclima.com/wp-content/uploads/Acclima-TDR-Sensor-User-Manual-REV-9-30-21.pdf) soil water content sensor. I set the address of the sensor to 1 and changed the serial port ID as you mentioned. I got records written in csv file, but unfortunatelly the records were all "unknow". I checked the entity state, it shows that this entity ("sensor.vwc_s1") does not have a unique ID. Trying adding the same unique id(s1) to each template sensor, it didn't work. It worked in the PC ternimal test
Can you please help me with advancing this?
HAha1

Hi there, looks like these sensors should indeed work as well. The output of those Acclima TDR looks a bit different from the Teros ones. I guess you’d first want to change the the way your sensor values are created.

In config.yaml you can see how I am splitting the whole SDI-12 output string into seperate sensor outputs. Since your SDI-12 output is different than mine, you might need to check how this SDI-12 output is split into seperate sensor’s (or entities). To learn more about splitting sensor values (or entity states), this post can help: Multiple ‘splits’ in value_template in sensor - #10 by HarmlessSaucer

In any case, the sensors should read ‘normal’ values in the entity’s state. After that, the Excel file will simply be a copy of those states.

I’m on a holiday for this week, so I can’t easily reproduce a working config for you. Let me know if you’re still unable to solve this next week, we can look into it.

Hi Rik, Thanks for your help. The output(1+0.0+21.0+1.1+0+0) of the Acclima TDR sensor is indeed very similar to the Teros sensor. I checked the link you directed and recheck the template sensor in the template editor of the HA. I checked the "sensor.serial_sensor" and "vwc_s1" sensor, both working well. However, when I run the automation, the output in the csv file only gave me the serial sensor not the vwc_s1 sensor which was shown "unknown". And another strange thing was sometimes when I restarted the HA server, the states output of the serial sensor is something like "100015" which I supposed the output of command "1M!" . I attached some output and the config & automation yaml below. looking forward to your further perspectives.

image


image
image

configuration.yaml
###################################################################
#                           SDI-12 Measure commands
###################################################################

switch:
  - platform: command_line
    switches:
      mm_sensor_1:
        command_on: 'echo "1M!" > /dev/serial/by-id/usb-FTDI_FT231X_USB_UART_D30A6XB3-if00-port0'    # Sends a measurement request to sensor 
        command_off: 'echo "1D0!" > /dev/serial/by-id/usb-FTDI_FT231X_USB_UART_D30A6XB3-if00-port0'    # Sends a measurement request to sensor 


###################################################################
#       Logging sensor outputs to a .CSV file and make Telegram messaging possible
###################################################################

notify:
  - platform: file
    name: filenotify
    filename: /config/soilwater.csv #Places measurements inside media/soilsensors folder
    timestamp: true


###################################################################
#                           Untemplated SDI-12 string
###################################################################

sensor:
  - platform: serial
    serial_port: /dev/serial/by-id/usb-FTDI_FT231X_USB_UART_D30A6XB3-if00-port0
    baudrate: 9600

  - platform: time_date
    display_options:
    - "date_time"

###################################################################
#                               SENSOR ADRESS
###################################################################

  - platform: template
    sensors:
      vwc_s1:
        friendly_name: Volumetric Water Content Sensor 1
        value_template: >-
          {% if "1" in states('sensor.serial_sensor')[0:1] %}
            {{ states('sensor.serial_sensor').split('+')[1] }}
          {% else %}
            {{ states('vwc_s1') }}
          {% endif %}

automations.yaml
###################################################################
#   Sending measurement commands and retrieve data commands to/from the sensor
###################################################################
- id: Log all sensors
  alias: Log all sensors
  trigger:
    platform: time_pattern
    seconds: '/10'             #Sends measurement requests every 60 seconds
#    minutes: '/15'              #Sends measurement requests every 15 minutes
  action:
# MAKE MEAUREMENTS FOR SENSOR 1
    - service: switch.turn_on
      entity_id: switch.mm_sensor_1
    - delay: 00:00:00.2  #Delay is in to enable the sensor to initiate the measurements and respond with information about itself and how many measurements it made
    - service: switch.turn_off
      entity_id: switch.mm_sensor_1
    - delay: 00:00:00.8  #Delay is in for the sensor time to respond with the actual sensor data
    - service: notify.filenotify #Log sensors data directly after measurement was retrieved
      data_template:
        message: "{{ states('sensor.serial_sensor') }}
                  {{ states('vwc_s1') }};"

Nice job on getting the sensors showing data, first thing I spotted is some missing info on both your configuration.yaml and the last line in your automation.yaml

Please redefine sensors from

{{ states(‘vwc_s1’) }}"

To

{{ states(‘sensor.vwc_s1’) }}"

When looking twice, I see you have pasted the vwc sensor into the serial sensor template. Please take a closer look at the example I gave in my starting post, you will actually need two seperate sensor templates: One for the serial sensor (that is outputting the unique sensor ID, and vwc or any other value you whish to extract from the SDI-12 string.

For the the serial sensor to output something like “100015” is a known issue I found a workaround for, I’ll share it with you once I have access to my current config (after holidays)

Many thanks. I finally got all sensors’ readings correct. The “10015” issue is still on, I supposed that the previous “unknown” issue was also derived from the wrong serial sensor output (“10015” ). Hope to ses your solution to this issue afterwards. Enjoy your holidays.
image

Here’s the workaround for ignoring those faulty values. This example works specifically for the Teros12 though, it is also a bit more complicated than just measuring a value. I’ll describe it shortly,

  • vwc_s1: The raw volumetric water content output, derrived from the SDI-12 string

  • c_vwc_s1: The actual calculated volumetric water content, based on equation that came with the Teros (you probably could give this a go). This sensor converts the raw output to a number between 0 and 1.

  • filtered_vwc_s1: The volumetric water content but filtered for negative values. The equation in ‘c_vwc_s1’ should always output a value between 0 and 1, everything else will result in ‘nan’ as output. The ‘nan’ value is conveniently ignored by graphs in home assistant.

      vwc_s1:
        friendly_name: Volumetric Water Content Sensor 1
        unit_of_measurement: "θ"
        value_template: >
          {% if '1' in states('sensor.serial_sensor') [0:1] -%}
            {{ states('sensor.serial_sensor').split('+')[1] | default}}
          {% else -%}
            {{ states.sensor.vwc_s1.state }}
          {% endif -%}

      c_vwc_s1:
        friendly_name: Calculated Volumetric Water Content Sensor 1
        unit_of_measurement: "θ"
        value_template: >
          {{ ((6.771e-10) * (states('sensor.vwc_s1')|float ** 3) -
             (5.105e-6) * (states('sensor.vwc_s1')|float ** 2) +
             (1.302e-2) * (states('sensor.vwc_s1')|float) -10.848) }}

      filtered_vwc_s1:
        friendly_name: Filtered Volumetric Water Content Sensor 1
        unit_of_measurement: "θ"
        value_template: >
          {%- if states.sensor.c_vwc_s1.state|float >= 0 and states.sensor.c_vwc_s1.state|float <= 1 -%}
            {{(states.sensor.c_vwc_s1.state) | float | round (3) }}
          {%- else -%}
            nan
          {%- endif -%}

Thanks, Rik. It has been long since I had put this project aside. I just checked your solution to the unknown issue, it works well. But another issue arises: the reading of the serial sensor never changed whatever I change the actual soil water content. The time_pattern I set up is every 60 seconds. See picture below. What do you think?
image

It looks like your sensor fails to make new measurements, but sends a cached measurement instead. Could you check the documentation of your sensor to see if the ‘make new measurement’ command is correct?

In other words, does command “1M!” make a new measurement and does command “1D0!” retrieve it?

-edit @jiangduo2013 I’m messaging you directly not to spoof the topic, I think we can figure out the solution and post it back here once resolved.

Yes. It’s the same.

Would it be difficult to make this work with the Teros 32 Tensiometer sensor? I’m working with an agronomist who recommends using both types of moisture sensors for my greenhouse operation.

The first issue I’m seeing is that barometric pressure is needed to calibrate one of the measurements reported by the sensor.
“TEROS 32 measures the sum of matric potential and atmospheric pressure potential (Ψp + Ψm ). To extract the matric potential, the barometric pressure should also be registered with a reference sensor. METER ZL6 and EM60 data loggers include a barometric pressure sensor and convert the signal into soil water potential If using a non-METER data logger, a barometric sensor is needed at the measuring site. A barometric sensor is available from METER by contacting Customer Support.”

I’m guessing I could use a temp/humidity sensor like the RuuviTag to get a pressure reading but no idea actually how to do that. I have very little coding experience and have mostly used node-red to create automations, while having HA handle all my BT sensors and smart plugs integrations. I have HA running 2 grow tents rn without touching my config or automation folder lol

If you know/have a working function for calibrating the TEROS32 against barometric pressure, the answer is -edit no: this should not be very difficult.

With a barometer sensor that is precise/reliable/fast (probably you want a dedicated SDI-12 barometer instead of something wireless) you can use that sensor reading in a template like i did in post 15. No hurt in testing this with a cheap barometer you have lying around first. If you have the calibration function, I can help you template the sensors.

hello everyone new member here so please be gentle!.

i have an acclima tdr water sensor one of liudrs sdi12 to usb adapters and a rp4 running homeassistant, @Potterstraat project and code is more or less exactly what i am wanting to acomplish with the only difference being i want to send and recieve the data wirelessly through an esp32 back to my rp4.

i have the physical connections all sorted using pins 16 + 17 (tx/rx) uart2 on the esp32 to the serial connector on dr lius sdi12 to usb board.
i am just unsure what lines of code need changing in the config.yaml to tell my esp32 to send and look for the data on the uart2 port and not the /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_AC00Z2T4-if00-port0 used for the usb connection.
can anyone help?, im a complete novice at this but willing and hopefully capable of learning.
many thanks.

So if I understand you right, you hooked the SDI-12 sensor up to an ESP32 and you want the ESP to talk to home assistant?

I think you can make this work with ESPHome: Custom UART Device — ESPHome

I guess you should be able to get the untemplated SDI-12 string into home assistant that way.

thanks for the speedy reply,

i have the sensor connected to the sdi12 to usb adapter and the adapter connected to the esp32 via uart.
and yes i would like the esp32 to send the read and retrieve data comands and to relay it back to homeassistant.

do you mean i should replace the original untemplated SDI-12 string code in your original config.yaml with the yaml code from the custom UART device link?.
like i said im a complete novice at this so please bare with me.

and what would need changing in the SDI-12 measure comands so it uses the uart for the on “1M!” and off “1D0!” and not the usb?.

also just to check is all this code going into the esp32’s config.yaml or does some go in the homeassistant config.yaml in file editor?.