Need precise seconds of switch ON state from history statistics

In short, I need to know how many millilitres my peristaltic pump (which pumps at a rate of 0.4166667 ml per second) has pumped a liquid at, so I can make some calculations on how long to run another peristaltic pump which controls the dosing of another additive at a later time for. And that’s where / why I need the accurate record of seconds for. :slight_smile:

BTW, I’m using PostgresQL and history statistics is enabled. The problem is the resolution it returns using history statistics sensor is only in hours and in decimal places of hours.

For instance:

  • platform: history_stats
    name: Reefdoser1 Pump3 Dosage Today
    entity_id: switch.reefdoser1_pump3
    state: ‘on’
    start: ‘{{ as_timestamp(now().replace(hour=0).replace(minute=0).replace(second=0)) }}’
    end: ‘{{ as_timestamp(now()) }}’

Only returns a value of precision 0.01 hours. And that provides only a resolution of 36 seconds.

What I really need is just the TOTAL number of seconds it was on. Not a ratio / decimal notation, which seems to be the default.

Thanks in advance!

you can adjust that and make it into a custom component with better resolution:

"""
Component to make instant statistics about your history.

For more details about this platform, please refer to the documentation at
https://home-assistant.io/components/sensor.history_stats/
"""
import datetime
import logging
import math

import voluptuous as vol

import homeassistant.components.history as history
import homeassistant.helpers.config_validation as cv
import homeassistant.util.dt as dt_util
from homeassistant.components.sensor import PLATFORM_SCHEMA
from homeassistant.const import (
    CONF_NAME, CONF_ENTITY_ID, CONF_STATE, CONF_TYPE,
    EVENT_HOMEASSISTANT_START)
from homeassistant.exceptions import TemplateError
from homeassistant.helpers.entity import Entity
from homeassistant.helpers.event import track_state_change

_LOGGER = logging.getLogger(__name__)

DOMAIN = 'history_stats'
DEPENDENCIES = ['history']

CONF_START = 'start'
CONF_END = 'end'
CONF_DURATION = 'duration'
CONF_PERIOD_KEYS = [CONF_START, CONF_END, CONF_DURATION]

CONF_TYPE_TIME = 'time'
CONF_TYPE_RATIO = 'ratio'
CONF_TYPE_COUNT = 'count'
CONF_TYPE_KEYS = [CONF_TYPE_TIME, CONF_TYPE_RATIO, CONF_TYPE_COUNT]

DEFAULT_NAME = 'unnamed statistics'
UNITS = {
    CONF_TYPE_TIME: 'h',
    CONF_TYPE_RATIO: '%',
    CONF_TYPE_COUNT: ''
}
ICON = 'mdi:chart-line'

ATTR_VALUE = 'value'


def exactly_two_period_keys(conf):
    """Ensure exactly 2 of CONF_PERIOD_KEYS are provided."""
    if sum(param in conf for param in CONF_PERIOD_KEYS) != 2:
        raise vol.Invalid('You must provide exactly 2 of the following:'
                          ' start, end, duration')
    return conf


PLATFORM_SCHEMA = vol.All(PLATFORM_SCHEMA.extend({
    vol.Required(CONF_ENTITY_ID): cv.entity_id,
    vol.Required(CONF_STATE): cv.string,
    vol.Optional(CONF_START): cv.template,
    vol.Optional(CONF_END): cv.template,
    vol.Optional(CONF_DURATION): cv.time_period,
    vol.Optional(CONF_TYPE, default=CONF_TYPE_TIME): vol.In(CONF_TYPE_KEYS),
    vol.Optional(CONF_NAME, default=DEFAULT_NAME): cv.string,
}), exactly_two_period_keys)


# noinspection PyUnusedLocal
def setup_platform(hass, config, add_devices, discovery_info=None):
    """Set up the History Stats sensor."""
    entity_id = config.get(CONF_ENTITY_ID)
    entity_state = config.get(CONF_STATE)
    start = config.get(CONF_START)
    end = config.get(CONF_END)
    duration = config.get(CONF_DURATION)
    sensor_type = config.get(CONF_TYPE)
    name = config.get(CONF_NAME)

    for template in [start, end]:
        if template is not None:
            template.hass = hass

    add_devices([HistoryStatsSensor(hass, entity_id, entity_state, start, end,
                                    duration, sensor_type, name)])

    return True


class HistoryStatsSensor(Entity):
    """Representation of a HistoryStats sensor."""

    def __init__(
            self, hass, entity_id, entity_state, start, end, duration,
            sensor_type, name):
        """Initialize the HistoryStats sensor."""
        self._hass = hass

        self._entity_id = entity_id
        self._entity_state = entity_state
        self._duration = duration
        self._start = start
        self._end = end
        self._type = sensor_type
        self._name = name
        self._unit_of_measurement = UNITS[sensor_type]

        self._period = (datetime.datetime.now(), datetime.datetime.now())
        self.value = None
        self.count = None

        def force_refresh(*args):
            """Force the component to refresh."""
            self.schedule_update_ha_state(True)

        # Update value when home assistant starts
        hass.bus.listen_once(EVENT_HOMEASSISTANT_START, force_refresh)

        # Update value when tracked entity changes its state
        track_state_change(hass, entity_id, force_refresh)

    @property
    def name(self):
        """Return the name of the sensor."""
        return self._name

    @property
    def state(self):
        """Return the state of the sensor."""
        if self.value is None or self.count is None:
            return None

        if self._type == CONF_TYPE_TIME:
            return round(self.value, 2)

        if self._type == CONF_TYPE_RATIO:
            return HistoryStatsHelper.pretty_ratio(self.value, self._period)

        if self._type == CONF_TYPE_COUNT:
            return self.count

    @property
    def unit_of_measurement(self):
        """Return the unit the value is expressed in."""
        return self._unit_of_measurement

    @property
    def should_poll(self):
        """Return the polling state."""
        return True

    @property
    def device_state_attributes(self):
        """Return the state attributes of the sensor."""
        if self.value is None:
            return {}

        hsh = HistoryStatsHelper
        return {
            ATTR_VALUE: hsh.pretty_duration(self.value),
        }

    @property
    def icon(self):
        """Return the icon to use in the frontend, if any."""
        return ICON

    def update(self):
        """Get the latest data and updates the states."""
        # Get previous values of start and end
        p_start, p_end = self._period

        # Parse templates
        self.update_period()
        start, end = self._period

        # Convert times to UTC
        start = dt_util.as_utc(start)
        end = dt_util.as_utc(end)
        p_start = dt_util.as_utc(p_start)
        p_end = dt_util.as_utc(p_end)
        now = datetime.datetime.now()

        # Compute integer timestamps
        start_timestamp = math.floor(dt_util.as_timestamp(start))
        end_timestamp = math.floor(dt_util.as_timestamp(end))
        p_start_timestamp = math.floor(dt_util.as_timestamp(p_start))
        p_end_timestamp = math.floor(dt_util.as_timestamp(p_end))
        now_timestamp = math.floor(dt_util.as_timestamp(now))

        # If period has not changed and current time after the period end...
        if start_timestamp == p_start_timestamp and \
            end_timestamp == p_end_timestamp and \
                end_timestamp <= now_timestamp:
            # Don't compute anything as the value cannot have changed
            return

        # Get history between start and end
        history_list = history.state_changes_during_period(
            self.hass, start, end, str(self._entity_id))

        if self._entity_id not in history_list.keys():
            return

        # Get the first state
        last_state = history.get_state(self.hass, start, self._entity_id)
        last_state = (last_state is not None and
                      last_state == self._entity_state)
        last_time = start_timestamp
        elapsed = 0
        count = 0

        # Make calculations
        for item in history_list.get(self._entity_id):
            current_state = item.state == self._entity_state
            current_time = item.last_changed.timestamp()

            if last_state:
                elapsed += current_time - last_time
            if current_state and not last_state:
                count += 1

            last_state = current_state
            last_time = current_time

        # Count time elapsed between last history state and end of measure
        if last_state:
            measure_end = min(end_timestamp, now_timestamp)
            elapsed += measure_end - last_time

        # Save value in hours
        self.value = elapsed / 3600

        # Save counter
        self.count = count

    def update_period(self):
        """Parse the templates and store a datetime tuple in _period."""
        start = None
        end = None

        # Parse start
        if self._start is not None:
            try:
                start_rendered = self._start.render()
            except (TemplateError, TypeError) as ex:
                HistoryStatsHelper.handle_template_exception(ex, 'start')
                return
            start = dt_util.parse_datetime(start_rendered)
            if start is None:
                try:
                    start = dt_util.as_local(dt_util.utc_from_timestamp(
                        math.floor(float(start_rendered))))
                except ValueError:
                    _LOGGER.error("Parsing error: start must be a datetime"
                                  "or a timestamp")
                    return

        # Parse end
        if self._end is not None:
            try:
                end_rendered = self._end.render()
            except (TemplateError, TypeError) as ex:
                HistoryStatsHelper.handle_template_exception(ex, 'end')
                return
            end = dt_util.parse_datetime(end_rendered)
            if end is None:
                try:
                    end = dt_util.as_local(dt_util.utc_from_timestamp(
                        math.floor(float(end_rendered))))
                except ValueError:
                    _LOGGER.error("Parsing error: end must be a datetime "
                                  "or a timestamp")
                    return

        # Calculate start or end using the duration
        if start is None:
            start = end - self._duration
        if end is None:
            end = start + self._duration

        self._period = start, end


class HistoryStatsHelper:
    """Static methods to make the HistoryStatsSensor code lighter."""

    @staticmethod
    def pretty_duration(hours):
        """Format a duration in days, hours, minutes, seconds."""
        seconds = int(3600 * hours)
        days, seconds = divmod(seconds, 86400)
        hours, seconds = divmod(seconds, 3600)
        minutes, seconds = divmod(seconds, 60)
        if days > 0:
            return '%dd %dh %dm' % (days, hours, minutes)
        elif hours > 0:
            return '%dh %dm' % (hours, minutes)
        return '%dm' % minutes

    @staticmethod
    def pretty_ratio(value, period):
        """Format the ratio of value / period duration."""
        if len(period) != 2 or period[0] == period[1]:
            return 0.0

        ratio = 100 * 3600 * value / (period[1] - period[0]).total_seconds()
        return round(ratio, 1)

    @staticmethod
    def handle_template_exception(ex, field):
        """Log an error nicely if the template cannot be interpreted."""
        if ex.args and ex.args[0].startswith(
                "UndefinedError: 'None' has no attribute"):
            # Common during HA startup - so just a warning
            _LOGGER.warning(ex)
            return
        _LOGGER.error("Error parsing template for field %s", field)
        _LOGGER.error(ex)

you’d be editing the pretty pretty_duration method of the HistoryStatsHelper class. You can have it default down to the .01 seconds.

2 Likes

Hi Petro,

Thanks for pointing me towards a very workable solution.

After going back and forth a bit through the history_statistics componenet, I finally figured it out and modified (delete, delete, delete…) the pretty_duration(hours) function to just return the total number of seconds, and it works.

For now, as you suggested, it’s an additional component to be called from my YAML config, but I’ll see about adding a proper time_seconds call of it’s own and submit it for a pull request. I came across a few other’s searching for this same problem, but no elegant solution and thinks this will do it for them. :smiley: It’s a huge win for me. Again, Thanks!

1 Like

No problem, glad to help. I don’t know why the guy made this have variable outputs that are controlled by how much time you use as an input. It doesn’t make sense. I would want the best resolution at all times, regardless of duration.

It does seem a little odd that s/he starts off manipulating / converting the very time notation I need (total seconds) into hours in decimal notion & ratio forms, but forgets/neglects to include an option for outputting in the total seconds (and possibly another option in total minutes). Looking at her/his code, it would have been easy to do with a few minutes of copy, paste & minor edits.
Funny omission is very funny. :wink:

hi.

how did you manage to change into seconds?

original code:

    def pretty_duration(hours)
        Format a duration in days, hours, minutes, seconds.
        seconds = int(3600  hours)
        if days  0
            return %dd %dh %dm % (days, hours, minutes)
        if hours  0
            return %dh %dm % (hours, minutes)
        return %dm % minutes 

what shall we delete?

tnx

In my opinion…Something like this is much better done on the switch device itself.
Then report the data to HA as soon as the event has completed.

If you were using a Tasmota switch you could do this with ‘rules’ I feel certain.

Something like this I think?
Rule 1 (triggered by the switch turning on) would start the timer and save the elapsed time to a variable.
On the switch turning off Rule 2 would ‘publish’ the elapsed time as a sensor value.

Thats the way I would try to tackle that anyway.

@novitibo

Make a custom_component from history_stats.py by copying it to history_stats_in_seconds.py

Then open history_stats_in_seconds.py and search for the following section and remark out:

    @staticmethod
    def pretty_duration(hours):
        """Format a duration in days, hours, minutes, seconds."""
        seconds = int(hours)
#        days, seconds = divmod(seconds, 86400)
#        hours, seconds = divmod(seconds, 3600)
#        minutes, seconds = divmod(seconds, 60)
#        if days > 0:
#            return '%dd %dh %dm' % (days, hours, minutes)
#        elif hours > 0:
#            return '%dh %dm' % (hours, minutes)
        return seconds

Then where you need the default behaviour, you can still call “history_stats” in your YAML config files, but when you need history in seconds, you just call “history_stats_in_seconds” in your platform: declaration.

Like this:

  - platform: history_stats_in_seconds
    name: Reefdoser1 Pump1 Dosage Today
    entity_id: switch.reefdoser1_pump1
    state: 'on'
    start: '{{ as_timestamp(now().replace(hour=0).replace(minute=0).replace(second=0)) }}'
    end: '{{ as_timestamp(now()) }}'
    unit_of_measurement: 's'
2 Likes

Unfortunately, that doesn’t work out nearly as accurate as the way I do it now. The reason for having history_stats_in_seconds custom component is so HASS can know in (near) real time when to flip the off switch. If for some reason, the switch doesn’t toggle off despite the command being sent, I then raise an alarm condition & take other automated actions.

For instance, I’ve already had it that a cat went mountaineering on the rack that holds my AquaDosers (controlled by the 4 way Sonoff) and activated one of the switches to a pump. In my setup now, my HASS saw this manual flip of the pump & shutdown the pump & raised an audible alarm to the Cat-ccident. :wink:

To achieve that the way you’re talking about, along with the flexibility of configuration & monitoring alarms, I think it would have to be a bit more complicated setup / code on both sides of the equation.

The way I have it now, let’s me get both pretty accurate ( can handle say just 68ml of dosing per day, split up 24 times in a single day) with just about +/- 1ml variance per day.

I have a total of 10 Sonoff controlled dosing pumps this way for a year, and been very pleased with the performance of it.

@cowboy Ok it sounds like you have developed a fairly comprehensive control system for what sounds like a very critical task there. Any chance of posting a photo of your dosing pump installation?
Do you have all your dosing equipment running on UPS protected supply as well?
What about your Hass server and network are they also on UPS protected supply?

Cheers for your reply!

Posted a whole HOWTO last year.

And the UPS answer and the reason I ranked it a lower priority is explained here. In short, it only covers one element of risk failure. So I embraced a solution that could cover for more types of failures than just power failures, offering a higher return on investment with a better fail-proof design.

Aha…I didn’t realise that was you…I have had a quick look at that project “Going to next level” but never really studied it all as it isn’t what I am doing…maybe I should have a better look I think.
Never seen the “Peristaltic” one before so I’ll study up on both of them.

Cheers!

how can I use the rules to get time how long the sensor measures energy > 0kWh?
need to calculate pellets loader consumption. grams/minute

tnx!

@novitibo In the rules document I believe there is an example which could be reworked to give you the time a sensor measures energy >0 I believe

The trigger would need to be in the form:

[SensorName]#[ValueName][comparison][value]


So ON [SensorName]#state>0 DO Var1 %timestamp% ENDON
And ON [SensorName]#state<1 DO Var2 %timestamp% ENDON

The final trigger and calculation of elapsed time I would need to spend some time on but note that the result is published so there is a great place to start.
EDIT: I think it needs to be based on %time% not %timestamp%. %time% is minutes from midnight.

ON mqtt#connected DO Publish stat/your_topic/your_sensor_name {"From":"%Var1%","To":"%Var2%"} ENDON

NOTE: That this is all ‘appended’ to be the one rule.

Example found at the bottom of the page here

ON wifi#disconnected DO Var1 %timestamp% ENDON
ON wifi#connected DO Var2 %timestamp% ENDON
ON mqtt#connected DO Publish stat/topic/BLACKOUT {"From":"%Var1%","To":"%Var2%"} ENDON

The final published ‘string’ just needs to be reworked to get a time value in minutes…not something I have done as yet but I would love to see what you come up with?

Hope that helps!

It works perfectly!
Tasmota rules:

Rule1 ON Energy#Power>0 DO Backlog Publish stat/loader/on {"Time":"%timestamp%"}; Rule2 1 ENDON 
Rule2 ON Energy#Power==0 DO Backlog Publish stat/loader/off {"Time":"%timestamp%"}; Rule2 0 ENDON

Home assistant sensors:

- platform: mqtt
  name: "loaderon"
  state_topic: "stat/loader/on"
  value_template: "{{value_json.Time}}"
  
- platform: mqtt
  name: "loaderoff"
  state_topic: "stat/loader/off"
  value_template: "{{value_json.Time}}"  
  
- platform: template
  sensors:
    loadertime:
      value_template: "{{ (as_timestamp(states.sensor.loaderoff.state)) | int  - (as_timestamp(states.sensor.loaderon.state)) | int}}"
      unit_of_measurement: "s"
  
- platform: history_stats
  name: loadertimeall
  entity_id: input_boolean.loadertime
  state: 'on'
  type: time
  start: '{{ 0 }}'
  end: '{{ now() }}' 

and automations:

- alias: loadertimeon
  trigger:
    platform: numeric_state
    entity_id: sensor.loadertime  
    below: 0         
  action:
    - service: input_boolean.turn_on
      entity_id: input_boolean.loadertime    
- alias: loadertimeoff
  trigger:
    platform: numeric_state
    entity_id: sensor.loadertime  
    above: 0    
  action:
    - service: input_boolean.turn_off
      entity_id: input_boolean.loadertime  

THANK U ALL!

2 Likes

@novitibo Well done!

I would be interested to see if the Tasmota device could do the maths as well though but your solution is neat.

Yeah, I tried with

{"From":"%Var1%","To":"%Var2%"}

but value of From and To did not include any timestamp. Any clue?

1 Like

@novitibo If I get time I will try this example you have working because there are actually a few areas I could probably use something like this. Cheers!

RE: this does not include values {“From”:"%Var1%",“To”:"%Var2%"}

Could you post the whole Rule you developed please?

In the meantime…If I find anything I will post again.

I used this alternative (splitted into two rules and then calculated in home assistant):

Rule1 ON Energy#Power>0 DO Backlog Publish stat/loader/on {"Time":"%timestamp%"}; Rule2 1 ENDON 
Rule2 ON Energy#Power==0 DO Backlog Publish stat/loader/off {"Time":"%timestamp%"}; Rule2 0 ENDON
1 Like

@novitibo yes I saw that above and thats what I am going to try out when I get time (I like how you use the rule to change whether the other rule is on or not as well).

But what I was looking for was how you were using the below snippet you mentioned?
And if you could post the whole rule it was in?

Cheers!