How to: Daily rain sensor from cumulative sensor using MQTT and template_sensor

phileep · June 21, 2017, 11:18pm

I have an RF weather sensor that gives me cumulative rainfall via MQTT since it was powered on.

I want to see the rainfall since midnight, so I use the following to store the rain to midnight yesterday as a reference value in mqtt (to ensure it survives any HASS restarts etc):

sensor:
    # Rain is cumulative forever - collect it and subtract yesterday for todays rain
  - platform: mqtt
    state_topic: 'RF/DKW2012-ID=004c'
    name: 'rain_cum'
    icon: mdi:weather-rainy
    unit_of_measurement: 'mm'
    value_template: '{{ value_json.RAIN }}'     
  - platform: mqtt
    state_topic: 'ha/cum_rain_prior'
    name: 'rain_cum_prior'
    unit_of_measurement: 'mm'
    value_template: '{{ value_json.day_1 }}' 
  - platform: template
    sensors:
      rain:
        value_template: '{%- if not (is_state("sensor.rain_cum","unknown") or is_state("sensor.rain_cum_prior","unknown") )-%}  {{ ((states.sensor.rain_cum.state | float) - (states.sensor.rain_cum_prior.state | float)) | max (0) | round(1) }} {%- endif -%}' ## ensure calc is no lower than zero!
        friendly_name: 'Rain Today'
        unit_of_measurement: 'mm'

Plus an automation to roll over the daily value after midnight:

automation:
  - alias: 'record cumulative rain to midnight'
    trigger:
      - platform: time
        at: "00:00:01"
    action:
      service: mqtt.publish
      data_template:
        topic: 'ha/cum_rain_prior'
        retain: true
        payload: '{"day_1":"{{states.sensor.rain_cum.state}}"}'

Gives me cumulative rain for the day ending midnight like so:

The templating took a bit of trial and error to ensure missing values were excluded from the maths … otherwise you can get negative rain at some points in the day if mqtt glitches!

Hope someone else finds this useful.

Phil

rpitera · June 22, 2017, 2:05pm

I put this in Share Your Projects so more people will see it. Nice solution!

bigwoof · October 16, 2017, 6:30am

Thanks Mate!!

It looks like I have the same weather station (I am using RFLink) and your config works great!

Now lots hope it keeps raining - we are currently on water restrictions here in Townsville (Australia).

PS: I think the time
after:
was removed since writing, as I needed to change it to:
at:

phileep · October 16, 2017, 8:48am

Good point about the automation trigger changing ~ version 0.50

Unfortunately I cant edit it - might be because it was moved by a dev.

mr.sneezy · March 5, 2018, 8:52am

Hi Phil.
I have the same protocol weather station it seems, as my RFlink hub detects it as DKW2012. The actual station physically is a Fine Offset WH2081 system though.
A question or two.
What model station do you have, and was MQTT native or something you added to it ?

The rain total data from the outdoor sensor on the WH2081 is an 8bit value from 0h to ffh on my station (I logged the data and checked that by manually tipping the bucket sensor) and it recycles to 0h after 76.8mm of rain (each bucket step = 0.3mm). The receiver console seems to check for 8bit rollover and increment the difference when that occurs and so keeps track of the accumulated total…

Have you got the same situation with your station rain register ?
Cheers,
Martin

phileep · March 5, 2018, 10:27am

Hi Martin,

I have a digitor weather station bought in Australia

I don’t have the rflink connected to my hass raspberry Pi directly using the hass component.

I use rflink on a mega but added mqtt via a esp8266 listening to the serial output from the mega. See my project here: Project: RFLink to MQTT using ESP8266 and arduino IDE

The esp sketch converts the data to decimal values and posts to mqtt. I then setup mqtt sensors in hass.

Phil

mr.sneezy · March 5, 2018, 11:34am

Yep I’m in Oz as well, Adelaide actually.
OK I’ll look into the esp8266 serial idea, seems like a good one and makes it more like the Sonoff RF Bridge. I have the Sonoff bridge too, but it doesn’t decode the weather station even with Espurna firmware, and of late it started to go nuts and trigger my 433Mhz gear randomly, so it’s been shutdown for now.

BTW, have you seen the high nibble of the rain counter ever increment ?
Just curious if that does it on yours, it should do it every 76.8mm of rainfall if that is an extension of the main rain counter byte.

phileep · March 5, 2018, 10:11pm

My cumulative rain is 1310.0 mm at the moment. Been in use for about 16 months.

I am not too concerned about cumulative as I really only track 24 hours blocks.

If it does roll to 0, I will have one day of weird reading, then the system will move ahead with daily delta from the rolled over total the next day.

A lot of my “rain” is actually wind shaking the bucket! Must secure my tv antenna better sometime! It’s not that wet in Perth!

mr.sneezy · March 6, 2018, 10:05am

Rain is a distant memory here in Adelaide since about last October really. Crazy wet over in the East though…

I did a bit of work on the WH2081 station tonight to try solve a few loose ends about the RF protocol. I had a low battery warning on the Fine Offset LCD console for some time (and the battery was two years old), but showing Batt OK on the HA GUI and RFlink debug. I logged the RF debug then changed the TX batteries and then captured a new log, sent off both to RFlink dev. The battery status cleared on the console of course, HA remained batt OK. I think the RFlink FW is reading the wrong status bit.

At the same time the total rain data from the TX cleared to 0h, both the high nibble and the main byte zeroed. I tipped the bucket twice and now the main byte shows 2h. So I’m fairly sure I was wrong and there is a high nibble for rain (so ranges to FFFh). No matter really, if you compare rain data every day unless you get 1228.5mm rain in that day and even that value rolls over

PS. I moved my rain sensor box down onto the main roof because of the shaking pole issue…

Naesstrom · April 12, 2018, 7:19pm

Any ideas if this would be usable for a sensor that’s not reporting by MQTT?
I’ve connected a Oregon PCR800 raingauge to my tellstick and I’m getting all the values to my home-assistant sensor sensor-rain_rateand sensor.rain_total. The total just keeps increasing and I want it to reset daily as yours do!

mr.sneezy · April 13, 2018, 8:13am

I haven’t found a nice neat understandable way yet either. Mine reports via the RFlink component in similar way to yours. I’d be happy with a way to reset a daily total at 9AM every day the same way the weather bureau does.

On the surface it seems easy, read and store the rain_total at 9AM, then displays the difference value increase during the day as a new entity, reset again next day at 9AM. The trick seems to be the way to create a ‘variable’ for the script to hold that 9AM capture…

phileep · April 14, 2018, 12:46am

@mr.sneezy, @Naesstrom

There are 3 sensors in my solution.

The first one (rain_cum) is from an MQTT rain sensor. That could be any sensor that holds cumulative values - change to the sensor you have.

The second one (rain_cum_prior) is the snapshot “variable” that we update manually in the automation. Even if your system does not use MQTT sensors, I would recommend using this as an MQTT sensor as it will store the value outside of HASS, so it will transcend restarts of HASS and remember the prior value. I am using mosquitto on the same pi as my HASS and this works fine for me, YMMV.

The third variable is just the delta between the other two - a common template sensor

So, @mr.sneezy, I suggest you just change your automation to take the snapshot at 0900 every day rather than midnight.

Hope this helps

Phil

mr.sneezy · April 14, 2018, 4:21am

Thanks phileep I might have enough understanding of HA config and MQTT to make sense now.
I must be blind though, I followed links back to your Git but did not see the yaml’s. Can you point me at them when you have time.

phileep · April 14, 2018, 4:29am

yaml extracts are in the first post in this thread - didn’t put in my github for the RFLink bridge as I recall

mr.sneezy · April 14, 2018, 4:59am

From my point of view there is black magic here still
The line
state_topic: ‘ha/cum_rain_prior’
creates a topic into HA itself that stores a value ?

phileep · April 15, 2018, 1:44am

Pretty much @mr.sneezy

The yaml you quoted tells HA where to look for the cumulative prior data - which is in MQTT at the topic ha/cum_rain_prior (you can name the topic whatever you want), I happen to prefix my homeassistant mqtt with ha (think of it kind of like a directory structure)

The “magic” is actually in the automation that runs at midnight (in my case). This publishes to the topic you are using as a sensor with the cumulative value at a point in time. Then the template sensor can observe the difference between the 2 to calculate period rainfall. When you publish to an MQTT topic, it overwrites any previous value that was there for exactly the same topic - so it is like a variable holder. By making the value in the MQTT topic retained, it will survive homeassistant reboots and whenever a device subscribes to this topic, mqtt will give it the last value.

You will need MQTT enabled to use this method (if you do not already have it enabled) - but it is baked in to homeassistant these days, so simply enabling it in your configuration.yaml should be enough - it is very a lightweight and straightforward protocol.

For first use, the cumulative sensor will not be populated - so either do this manually from hass or command line … or the easy option is to manually trigger the automation to do the cumulative update. This would make prior and current cumulative the same, so expect 0 rainfall until the bucket gets more through it!

mr.sneezy · April 15, 2018, 7:29am

Do you write training material or teach for a living ? That’s one of the best short descriptive replies I’ve read here.
Thanks very much for it.

It had not occurred to me that an MQTT topic was really so abstract.

I’ll try this out shortly.
Martin

phileep · April 15, 2018, 12:47pm

Glad it helped

Hope you get it to work.

Once you get your head around it, you see why MQTT is so pervasive … it just works!!

mr.sneezy · May 18, 2018, 12:47pm

Hi phileep.
I’m back onto tinkering with my HA setup again and a daily rain counter.

I’ve tried hard tonight to get the MQTT based daily rain sensor configurations to work but not had success yet.
After some fine tuning I got the config and automations yaml’s to pass a user initiated configuration validation test. However when attempting a HA restart to try the new stuff I get another validation error warning on the Home screen.

The error log seems to hold this info:
Log Details (ERROR)
Fri May 18 2018 21:58:51 GMT+0930 (Cen. Australia Standard Time)

starting version 3.2.4
Testing configuration at /config
ERROR:homeassistant.scripts.check_config:BURB
Traceback (most recent call last):
File “/usr/lib/python3.6/site-packages/homeassistant/scripts/check_config.py”, line 207, in check
res[‘components’] = check_ha_config_file(hass)
File “/usr/lib/python3.6/site-packages/homeassistant/scripts/check_config.py”, line 328, in check_ha_config_file
del core_config[CONF_PACKAGES]
KeyError: ‘packages’
Fatal error while loading config: ‘packages’
Failed config
General Errors:
- ‘packages’
Successful config (partial)

My cumulative rain sensor come via the RFlink component not via MQTT and the sensor ID is sensor.dkw2012_00f7_raintot
This is my configuration block:

  - platform: mqtt
    state_topic: 'ha/cum_rain_prior'
    name: 'rain_cum_prior'
    unit_of_measurement: 'mm'
    value_template: '{{ value_json.day_1 }}' 
  - platform: template
    sensors:
      rain:
        value_template: '{%- if not (is_state("sensor.dkw2012_00f7_raintot","unknown") or is_state("sensor.rain_cum_prior","unknown") )-%}  {{ ((states.sensor.dkw2012_00f7_raintot.state | float) - (states.sensor.rain_cum_prior.state | float)) | max (0) | round(1) }} {%- endif -%}' ## ensure calc is no lower than zero!
        friendly_name: 'Rain Today'
        unit_of_measurement: 'mm'

This is my automation block

- alias: 'record cumulative rain to midnight'
  trigger:
    platform: time
    at: '09:00:00'
  action:
    service: mqtt.publish
    data_template:
      topic: 'ha/cum_rain_prior'
      retain: true
      payload: '{"day_1":"{{states.sensor.dkw2012_00f7_raintot.state}}"}'

Hoping somebody can see something I’ve got wrong or omitted.

phileep · May 21, 2018, 4:05am

Hey @mr.sneezy

Sorry for slow reply - Have been away.

Do you have MQTT enabled in HASS?

You need an MQTT section in configuraton.yaml before trying to use the mqtt sensors etc: https://www.home-assistant.io/components/mqtt/

Phil