How to best run sensor reading the output of a Python script?

Tags: #<Tag:0x00007f32706806d8> #<Tag:0x00007f3270680610>

Hi

I am new to HA. I have installed hass.io in a VM on Windows Server and everything is running. I have ssh access to the Home Assistant shell. I’m a little confused with hass.io/supevisor/ha itself and this is the first time I’ve seen yaml.

I would like to bring in some air quality information as a sensor into HA. I have a local station that publishes the raw pollutant data and I have written a short Python script to scrape it and process it to calculate some indices (it’s a piecewise linear function with some choice of maximal params). I can output this to the shell or a text file, or whatever is best. One thing to note is that I have multiple data from each scrape (levels of different pollutants) and I would like to feed them to different sensors (attributes?) in HA.

I would like to understand a hassionic way of bringing this in as a sensor. I can see a few scenarios:

  • Scrape directly in HA through the scrape sensor and replicate the mathematical operations in HA. Not sure if this is possible/appropriate in YAML. Is there some better method?

  • use the command_line sensor to run the python script and get its output. I don’t see how to get multiple sensors out without running the scrape multiple times and filtering the output differently for each sensor. Might get annoying for the data server.

  • Run the python script as a cron job every 15 mins, outputting to a text file and use the file as a data source for the sensor. Where would I set up cron? It doesn’t seem available in the HA shell, Maybe it needs to be at a lower level in the VMs? How do I access it?

Then there are some things in HACS called appdaemon and netdaemon: may this is a better way of achieving what I want? Haven’t investigated that part of HA yet.

But I may be barking up the wrong tree completely. Advice much appreciated!

Thanks!

I’m a fan of publishing to an mqtt broker and subscribing to the topics in Home Assistant (mqtt sensor).

Other options:

Push the values from your script directly to Home Assistant using the Home Assistant rest API ( e.g. post /api/states/<entity_id>).

Skip the Python script and scrape directly using home assistant, (scrape sensor)

why not create the sensor in the Python script directly? You can set the state and all attributes.

Could you please point me to some documentation/examples? I have no real idea of the architecture of HA yet.

Also how do I run the script in HA? in the HA shell cron doesn’t seem available?

How? Because maybe you could replace your Python script by a scrape or RESTful sensor?

There is no API at the weather station, so I have to scrape the data from a table on a webpage.

The problem is that the data are raw quantity measurements and I have to apply some simple arithmetic to get the index I want to present in the Dashboard.

Can I do algebraic transformations on scraped data within HA? It’s just a bunch of min/max functions and multiplying by constants, so it’s not very complicated, but I don’t see where I could do it.

Algebraic operations can be done using Template Sensors or predefined Min/Max or Statistics sensors, that consume other sensor’s values (like your scrape sensor).

read up on te subject: https://www.home-assistant.io/integrations/python_script/

here’s a simple script, python_script.overview_entities:

count_all = 0
domains = []
attributes = {}

for entity_id in hass.states.entity_ids():
    count_all = count_all + 1
    entity_domain = entity_id.split('.')[0]
    if entity_domain not in domains:
        domains.append(entity_domain)

attributes['Domains'] = len(domains)
attributes['--------------'] = '-------'

for domain in sorted(domains):
    attributes[domain] = len(hass.states.entity_ids(domain))

attributes['friendly_name'] = 'Entities'
attributes['icon'] = 'mdi:format-list-numbered'

hass.states.set('sensor.overview_entities', count_all, attributes)

you can run a python script with the service python_script.name_of_the_script which will show up after you have saved the script, and reloaded the python scripts with the service python_script.reload

use the service in a HA script or automation:

  run_after_delayed_startup:
    alias: Run after delayed startup
    mode: restart
    sequence:
      - service: python_script.average_indoor_temp
# give even extra time for system to settle before doing:
      - delay:
          seconds: >
            {{states('input_number.ha_delayed_startup')|int}}
      - service: python_script.overview_entities
      - service: python_script.overview_components
      - service: script.update_entities_uun

a couple of other examples:

  - alias: Last Automation
    id: Last Automation
    trigger:
      platform: event
      event_type: automation_triggered
    condition: []
    mode: queued
    max: 50
    action:
      service: python_script.last_automation
      data:
        event: >
          {{trigger.event}}

or

  - alias: Run at startup
    id: Run at startup
    trigger:
      platform: homeassistant
      event: start
    action:
      - service: script.notify_startup
      - service: python_script.family_home
      - delay:
          seconds: >
            {{states('input_number.ha_delayed_startup')|int}}
      - event: delayed_homeassistant_start

Thanks for the pointers. So I have studied the Jinja2 template language and it turns out it’s quite flexible. In case anyone else is looking a this in the future, I leave here my implementation of a simple piecewise linear conversion of pollutant concentration to a CAQI subindex as a macro (function) in the template language

caqi_so3:
                name: CAQI SO3
                unit_of_measurement: " "
                # levels: 0, 50, 100, 350, 500
                select: "#content > table >tr:nth-child(27) > td:nth-child(3)"
                value_template: >
                    {% macro calc_caqi(val, range_top, range_bot, caqi_top, caqi_bot) -%}
                    {{ '%0.1f'|format(caqi_bot + (caqi_top-caqi_bot)/(range_top-range_bot)*(val-range_bot)) }}
                    {%- endmacro %}
                    {% set levels = [500,350,100,50,0] %}
                    {% set caqi_levels = [100,75,50,25,0] %}
                    {% set sensor = float(value) %}
                    {% if sensor >levels[0] -%}
                    {{'%0.1f'|format(caqi_levels[0] + (caqi_levels[0]-caqi_levels[1])/(levels[0]-levels[1])*(val-levels[1])) }}
                    {%- elif sensor >= levels[1] -%}
                    {{calc_caqi(sensor,levels[3],levels[4],caqi_levels[3],caqi_levels[4]) }}
                    {%- elif sensor >= levels[2] -%}
                    {{calc_caqi(sensor,levels[1],levels[2],caqi_levels[1],caqi_levels[2]) }}
                    {%- elif sensor >= levels[3] -%}
                    {{calc_caqi(sensor,levels[2],levels[3],caqi_levels[2],caqi_levels[3]) }}
                    {%- elif sensor >= levels[4] -%}
                    {{calc_caqi(sensor,levels[3],levels[4],caqi_levels[3],caqi_levels[4]) }}
                    {%- else -%}
                    N/A
                    {%- endif %}