Custom Components Throttle functionality etc

Hi All,

Totally new to everythine here, so my apologies if some of these questions are too … newbie? :slight_smile:
I started using Home Assistant a couple of weeks ago after seeing its capabilities. My home automation system so far consisted of Insteon (ISY994) together with a host of sensors and other items that I coded in B&R PLC code, C++ and C# over the years. The capability of Home Assistant to bring this together in one platform is enticing to say the least!
I’m also totally new to Python and YAML, even though I have coded my entire adult life (but all in C++/C#). I have to say that it’s a little tough to get used to spacing in Python instead of brackets. :slight_smile:

Anyway, in order to get any existing sensors and equipment merged into Home Assistant, I created a few custom components. Starting with modifying the existing TED5000 (power meter) component to work with my newer TED Pro with Spyders. That went fairly smoothly, so my next step was integrating my well sensor (Wellntel), which also wasn’t too big of a problem … But before I get my entire system over to a more native HomeAssistant format, I still have about 40-some legacy sensors that are broadcasting their information across my network in one aggregated XML UDP message every 1s or so. So I created a custom component to pick up that XML UDP, decode it, and make sensors out of them in the home assistant platform.
So I have one custom component that registers a bunch of sensors like such:
dev.append(Sensor(recv, "Floor Heat Zone 1 Temperature", TEMP_FAHRENHEIT, 42, "vdb_floor_heat_zone_1_temp")), etc, and finally a add_entities(dev) call.
Then I have an update method that listens for the UDP message, decodes it, and puts it in a class variable. The state call pulls out the exact one that HomeAssistant is looking for and sends it back.
So far, so good … But I had to do some creative coding to prevent HomeAssistant from calling update for every instance created (I really only need one update call for all my states).
What is the proper way to do that or structure that? For now, I just return from my update method immediately when it’s called in < 10s (my normal update rate). That works, but seems ugly. I have seen some people use “throttle”, but couldn’t figure out if that just queued up all the update calls before the state calls, or if it would do pretty much what I coded…

I’m sure there are better ways to handle this, but with my limited Python and HomeAssistant experience, I’m just starting out …
In C++ I would put the UDP stuff in a thread that listens, and have state calls pull from a dictionary or similar type. I tried to code that in Python, but couldn’t figure out how I would have Home Assistant abort the thread safely when it wants to (when would that be? what would it call?). And since I didn’t want my home assistant instance to freeze or cause issues, I only tried this in python on my local system. :slight_smile:

Any guidance / info is welcome! Again – for now it seems to work, but it just feels … ugly. :slight_smile:

Looks like you’ve accomplished a lot already!

I’d suggest you have a look here: https://developers.home-assistant.io/docs/development_index/
Also, as an alternative, you could create custom_components: https://aarongodfrey.dev/home%20automation/building_a_home_assistant_custom_component_part_1/
and the further parts.

Happy reading!

Thanks for the suggestions. I did find those resources as well, but I tend to be a trial & error coder more than a sit-down and read for 2 days and then code coder…
When I went looking specifically for what the throttle function did, I couldn’t find anything in there, or on google.
The pattern I found used in the ted5000 integration that is built-in to Home Assistant was this:

from homeassistant.util import Throttle
[…]
And then right before the update method:
@Throttle(MIN_TIME_BETWEEN_UPDATES)
def update(self):

Is there any documentation specific to this anywhere? And to be clear – I’d like to understand what Throttle does exactly. I can see the following potential ways it would work:

  1. Returns without doing any action in the update method (if the call is done within the throttle time) and still calls the state method for that sensor.
  2. Returns without doing any action in the update method (if the call is done within the throttle time) and also doesn’t call the state method for that sensor.
  3. Queues up the calls to update and state for that sensor and executes them when the throttle time is “up”.

For my usage, “1” would be the best, but I suspect “2” or “3” is happening …

Have you looked at the source code: https://github.com/home-assistant/core/blob/8fe51b8ea7e5912a70f3650bd0fb794681e67524/homeassistant/util/init.py
Look at the class Throttle.

Robbet,

I have a TED with a spyder that I would like to get integrated also. Would you mind sharing the changes you made to the component?

Hi Andre –

Sure … It’s a total hack-job though. :slight_smile: Here’s the process:
Copy the files from the ted5000 component into the custom_components folder – create a new one called “tedpro” or something like that.
Modify the init.py to the following:
"""The ted pro component."""

Modify the manifest.json to the following:
{
"domain": "tedpro",
"name": "The Energy Detective Pro",
"documentation": "https://www.home-assistant.io/integrations/ted5000",
"requirements": ["xmltodict==0.12.0"],
"codeowners": []
}

And then edit the sensor.py to the following:

"""Support gathering ted pro information."""
from datetime import timedelta
import logging

import requests
import voluptuous as vol
import xmltodict

from homeassistant.components.sensor import PLATFORM_SCHEMA
from homeassistant.const import CONF_HOST, CONF_NAME, CONF_PORT, POWER_WATT, VOLT, ELECTRICAL_VOLT_AMPERE, PERCENTAGE
from homeassistant.helpers import config_validation as cv
from homeassistant.helpers.entity import Entity
from homeassistant.util import Throttle

_LOGGER = logging.getLogger(__name__)

DEFAULT_NAME = "ted"

MIN_TIME_BETWEEN_UPDATES = timedelta(seconds=2)


PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend(
    {
        vol.Required(CONF_HOST): cv.string,
        vol.Optional(CONF_PORT, default=80): cv.port,
        vol.Optional(CONF_NAME, default=DEFAULT_NAME): cv.string,
    }
)


def setup_platform(hass, config, add_entities, discovery_info=None):
    """Set up the Ted pro sensor."""
    host = config.get(CONF_HOST)
    port = config.get(CONF_PORT)
    name = config.get(CONF_NAME)
    url = f"http://{host}:{port}/api/SystemOverview.xml"
    urlSpyder = f"http://{host}:{port}/api/SpyderData.xml"

    gateway = TedProGateway(url, urlSpyder)

    # Get MUT information to create the sensors.
    gateway.update()

    dev = []
    for mtu in gateway.data:
        dev.append(TedProSensor(gateway, name, mtu, POWER_WATT))
        dev.append(TedProSensor(gateway, name, mtu, VOLT))
        dev.append(TedProSensor(gateway, name, mtu, ELECTRICAL_VOLT_AMPERE))
        dev.append(TedProSensor(gateway, name, mtu, PERCENTAGE))
    # Individual sensors (derived)
    dev.append(PowerSensor(gateway, "Power Grid", 0, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Solar", 1, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power ADU", 2, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Well Pump", 3, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Fridge", 4, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power UPS", 5, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Dishwasher", 6, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Washer", 7, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Microwave", 8, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Dryer", 9, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Outdoor", 10, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Car Charger", 11, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Sunroom Floor", 12, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Finished Basement", 13, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Unfinished Basement", 14, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power AC Air Handler", 15, POWER_WATT))
    dev.append(PowerSensor(gateway, "Power Stove", 16, POWER_WATT))

    add_entities(dev)
    
    return True


class TedProSensor(Entity):
    """Implementation of a Ted Pro sensor."""

    def __init__(self, gateway, name, mtu, unit):
        """Initialize the sensor."""
        units = {POWER_WATT: "power", VOLT: "voltage", ELECTRICAL_VOLT_AMPERE: "kva", PERCENTAGE: "pf"}
        self._gateway = gateway
        self._name = "{} mtu{} {}".format(name, mtu, units[unit])
        self._mtu = mtu
        self._unit = unit
        self.update()

    @property
    def name(self):
        """Return the name of the sensor."""
        return self._name

    @property
    def unit_of_measurement(self):
        """Return the unit the value is expressed in."""
        return self._unit

    @property
    def state(self):
        """Return the state of the resources."""
        try:
            return self._gateway.data[self._mtu][self._unit]
        except KeyError:
            pass

    def update(self):
        """Get the latest data from REST API."""
        self._gateway.update()

class PowerSensor(Entity):
    """Implementation of a Ted Pro sensor."""

    def __init__(self, gateway, name, number, unit):
        """Initialize the sensor."""
        units = {POWER_WATT: "power"}
        self._gateway = gateway
        self._name = name
        self._number = number
        self._unit = unit
        self.update()

    @property
    def name(self):
        """Return the name of the sensor."""
        return self._name

    @property
    def unit_of_measurement(self):
        """Return the unit the value is expressed in."""
        return self._unit

    @property
    def state(self):
        """Return the state of the resources."""
        try:
            return self._gateway.power[self._number]
        except KeyError:
            pass

    def update(self):
        """Get the latest data from REST API."""
        self._gateway.update()


class TedProGateway:
    """The class for handling the data retrieval."""

    def __init__(self, url, urlSpyder):
        """Initialize the data object."""
        self.url = url
        self.urlSpyder = urlSpyder
        self.data = {}
        self.power = {}

    @Throttle(MIN_TIME_BETWEEN_UPDATES)
    def update(self):
        """Get the latest data from the TedPro XML API."""

        try:
            request = requests.get(self.url, timeout=10)
            requestSpyder = requests.get(self.urlSpyder, timeout=10)
        except requests.exceptions.RequestException as err:
            _LOGGER.error("No connection to endpoint: %s", err)
        else:
            doc = xmltodict.parse(request.text)
            docSpyder = xmltodict.parse(requestSpyder.text)
            mtus = 3 
            """int(doc["DialDataDetail"]["MTUVal"]["NumberMTU"])"""

            for mtu in range(1, mtus + 1):
                power = int(doc["DialDataDetail"]["MTUVal"]["MTU%d" % mtu]["Value"])
                voltage = int(doc["DialDataDetail"]["MTUVal"]["MTU%d" % mtu]["Voltage"])
                kva = int(doc["DialDataDetail"]["MTUVal"]["MTU%d" % mtu]["KVA"])
                pf = int(doc["DialDataDetail"]["MTUVal"]["MTU%d" % mtu]["PF"])

                self.data[mtu] = {POWER_WATT: power, VOLT: voltage / 10, ELECTRICAL_VOLT_AMPERE: kva, PERCENTAGE: pf / 10}
            
            mtu1power = int(doc["DialDataDetail"]["MTUVal"]["MTU1"]["Value"])
            mtu2power = int(doc["DialDataDetail"]["MTUVal"]["MTU2"]["Value"])
            mtu3power = int(doc["DialDataDetail"]["MTUVal"]["MTU3"]["Value"])

            gridpower = mtu1power-mtu3power
            solarpower = -mtu2power
            adupower = mtu3power

            # Set up individual power sensors
            self.power[0] = gridpower     #grid minus ADU
            self.power[1] = solarpower    # Solar
            self.power[2] = mtu3power     # ADU

            # Parse through Spyder data
            spyder1 = docSpyder["SpyderData"]["Spyder"][0]
            spyder2 = docSpyder["SpyderData"]["Spyder"][1]

            self.power[3] = int(spyder1['Group'][0]['Now'])
            self.power[4] = int(spyder1['Group'][1]['Now'])
            self.power[5] = int(spyder1['Group'][2]['Now'])
            self.power[6] = int(spyder1['Group'][3]['Now'])
            self.power[7] = int(spyder1['Group'][4]['Now'])
            self.power[8] = int(spyder1['Group'][5]['Now'])
            self.power[9] = int(spyder1['Group'][6]['Now'])
            self.power[10] = int(spyder1['Group'][7]['Now'])

            self.power[11] = int(spyder2['Group'][0]['Now'])
            self.power[12] = int(spyder2['Group'][1]['Now'])
            self.power[13] = int(spyder2['Group'][2]['Now'])
            self.power[14] = int(spyder2['Group'][3]['Now'])
            self.power[15] = int(spyder2['Group'][4]['Now'])
            self.power[16] = int(spyder2['Group'][5]['Now'])

You will need to modify the Spyder sensors in dev.append in the setup_platform method. Plus I’m sure that if you’re a Python programmer, you can probably figure out a much better way to do this.

I’m basically mapping a number in the self.power[] array to a sensor.

You’ll also need to change the # of mtu’s to match your setup. I have 3 mtu’s. I also do some fancy math with the raw power data from the mtu’s to make it make more sense with my particular setup. That’s happening in the self.power[0] … code.

To make this work, you will need to add:
- platform: tedpro
host: <ip address of your tedpro

to your configuration.yaml.

Again – this is a total hack-job. It’s not on Git, because I frankly don’t have the time to make this bullet proof and universal for everyone to use. Feel free to grab my mods though (for anyone wanting to do that). :slight_smile:

Good luck!

I am not much of a programmer at all. My install is similar in that I have a grid and solar MTU. Thanks for the code. I was able to get it up and running. I might be able follow what you have done to try and get my hacked Lutron Grafik Eye integration in a better state also.

After I got the “hang” of it, it wasn’t too difficult to follow Python code. Glad to hear you got it working!
I also added my Wellntel well depth sensor and custom home automation in the same manner. It’s been working great for the past week or so, so I think the code is ok. :slight_smile:

@rvdbijl I was surprised to see Wellntel mentioned in a post on Home Assistant.
Would you be willing to share your progress on incorperating its data into Home Assistant?
I would love to incorperate it, along with my propane sensors (ESP32 in the mail).

I too complained about Python spacing at first. Slowly but surely got used to it.

Thanks in advance.
Dennis

Hi Dennis –

Yes, my Wellntel is fully functional, but I did need an API key from them to make it work. They were a bit concerned about hits to their server, so I lowered the rate to once every 5 mins (down from once per minute).
With Wellntel as a company seeming less and less interested in the single consumer, as soon as something breaks or they raise their API rates further, I’m switching to one of these: Well Watch 670 - Eno Scientific. From what I understand, they use BLE to transmit the signal to an indoor gauge, so perhaps I could intercept that. If not, it’s easy enough to stash a ESP in there with an analog (or serial) input.

Code below:

from datetime import timedelta
import logging

import requests
import voluptuous as vol
import json

from homeassistant.components.sensor import PLATFORM_SCHEMA
from homeassistant.const import CONF_API_KEY, CONF_NAME, ELECTRIC_POTENTIAL_VOLT, LENGTH_FEET, SIGNAL_STRENGTH_DECIBELS, TEMP_FAHRENHEIT
from homeassistant.helpers import config_validation as cv
from homeassistant.helpers.entity import Entity
from homeassistant.util import Throttle

_LOGGER = logging.getLogger(__name__)

DEFAULT_NAME = "Wellntel"

MIN_TIME_BETWEEN_UPDATES = timedelta(seconds=300)


PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend(
    {
        vol.Required(CONF_API_KEY): cv.string,
        vol.Optional(CONF_NAME, default=DEFAULT_NAME): cv.string
    }
)


def setup_platform(hass, config, add_entities, discovery_info=None):
    """Set up the Wellntel sensor."""
    name = config.get(CONF_NAME)
    apikey = config.get(CONF_API_KEY)
    url = f"https://connect.wellntel.com/analytics-api/readings?count=1&order=descending"

    gateway = WellntelAPI(url, apikey)

    # Get MUT information to create the sensors.
    gateway.update()

    dev = []
   
    dev.append(WellSensor(gateway, "Wellntel Depth", LENGTH_FEET))
    dev.append(WellSensor(gateway, "Wellntel RSSI", SIGNAL_STRENGTH_DECIBELS))
    dev.append(WellSensor(gateway, "Wellntel Battery Voltage", ELECTRIC_POTENTIAL_VOLT))
    dev.append(WellSensor(gateway, "Wellntel Temperature", TEMP_FAHRENHEIT))

    add_entities(dev)
    
    return True

class WellSensor(Entity):
    """Implementation of a Wellntel sensor."""

    def __init__(self, gateway, name, unit):
        """Initialize the sensor."""
        units = {LENGTH_FEET: "depth", SIGNAL_STRENGTH_DECIBELS: "rssi", ELECTRIC_POTENTIAL_VOLT: "voltage", TEMP_FAHRENHEIT: "temp"}
        self._gateway = gateway
        self._name = name
        self._unit = unit
        self.update()

    @property
    def name(self):
        """Return the name of the sensor."""
        return self._name

    @property
    def unit_of_measurement(self):
        """Return the unit the value is expressed in."""
        return self._unit

    @property
    def state(self):
        """Return the state of the resources."""
        try:
            return self._gateway.data[self._unit]
        except KeyError:
            pass

    def update(self):
        """Get the latest data from REST API."""
        self._gateway.update()


class WellntelAPI:
    """The class for handling the data retrieval."""

    def __init__(self, url, apikey):
        """Initialize the data object."""
        self.url = url
        self.apikey = apikey
        self.data = {}

    @Throttle(MIN_TIME_BETWEEN_UPDATES)
    def update(self):
        """Get the latest data from the TedPro XML API."""

        try:
            request = requests.get(self.url, headers={'Authorization': 'Key '+self.apikey}, timeout=10)
            wellnteldata = json.loads(request.text)
        except requests.exceptions.RequestException as err:
            _LOGGER.error("No connection to endpoint: %s", err)
        else:
            depth = -wellnteldata[0]['depth']
            rssi = wellnteldata[0]['rssi']
            voltage = wellnteldata[0]['batteryvoltage']/10
            temp = wellnteldata[0]['temperature']/10
            
            self.data = {LENGTH_FEET: depth, SIGNAL_STRENGTH_DECIBELS: rssi, ELECTRIC_POTENTIAL_VOLT: voltage, TEMP_FAHRENHEIT: temp}