Curl > json command

Configuration
49

of course! duh. Thank you! no querying the Zwave hub required, just subscribing to the published events…

check this:
regular mqtt sensors reading property: State

09
09933×122 45 KB

binary sensors reading the same property: State

07
07942×109 40.3 KB

this might even be better than the command_line version, since we can now use the Qos=1, making sure the message is red and delivered. Before, quite often the command wouldn’t stick because of several intermediate issues (timing, availability, sequence etc etc)

I am still a bit puzzled how to combine these into one template switch though, please give me a nudge:

this is the code i use now for a regular mqtt sensor:

- platform: mqtt
  state_topic: 'mac_address/powerswitch-zwave/4e6eeb01/state'
  name: "Tester state"
  unit_of_measurement: "State"
#  value_template: "{{ value | round(2) }}"

the code for the binary sensor:

- platform: mqtt
  name: "Tester state bin"
  state_topic: 'mac_address/powerswitch-zwave/4e6eeb01/state'
  payload_on: "on"
  payload_off: "off"
#  availability_topic: "home-assistant/window/availability"
#  payload_available: "online"
#  payload_not_available: "offline"
  qos: 1
  device_class: plug
#  value_template: '{{ value.x }}'

And the current Command_line Switch:

sw_tester_cl:
  friendly_name: 7 - Tester
  command_on: >-
    curl -X POST -d '{"seq":1, "method":"object_prop_set", "arguments":{"oid":"4e6eeb01", "prop":"command", "value":"on"}}' http://192.168.xxx.xxx/iungo/api_request
  command_off: >-
    curl -X POST -d '{"seq":1, "method":"object_prop_set", "arguments":{"oid":"4e6eeb01", "prop":"command", "value":"off"}}' http://192.168.xxx.xxx/iungo/api_request#      command_state: >-
  command_state: >-
    curl -X POST -d '{"seq":1, "method":"object_prop_get", "arguments":{"oid":"4e6eeb01", "prop":"state"}}' http://192.168.xxx.xxx/iungo/api_request
  value_template: >-
    {%- if value_json.rv.value == 'on' -%}
    {{ true }}
    {%- else -%}
    {{ false }}
    {%- endif -%}

trying this preliminary code for starters:

- platform: template
  switches:
    switch_template_tester:
      friendly_name: "Tester"
      value_template: "{{ is_state('binary_sensor.tester_state_bin.state', 'on') }}"
      turn_on:
        service: switch.turn_on
        data:
          entity_id: switch.sw_tester_cl
      turn_off:
        service: switch.turn_off
        data:
          entity_id: switch.sw_tester_cl

Thanks!

50

Im not sure where to start.
my home-assistant_v2.db is only 111,5 mb. I have MariaDB installed but just admit i don’t have any idea where it is stored on the flash card…
Other than a few backups (max 3gb) and the system files, i wouldn’t know where to find the space-hogs.

Ive tried to limit the logging by entering the following in my logbook.yaml, so it doesn’t even log the sensors yet…:
include:
domains:
- script
- notify
- switch
entities:
- device_tracker.phone1,2,3,4,5,6

51

HI @NotoriousBDG,

Your advise was very appreciated, ive made the switches based upon the above, with a slight change in the state sensor:

    sw_tester_template:
      friendly_name: "7 - Tester"
      value_template: "{{ is_state('binary_sensor.tester_state_bin', 'on') }}"
      turn_on:
        service: switch.turn_on
        data:
          entity_id: switch.sw_tester_cl
      turn_off:
        service: switch.turn_off
        data:
          entity_id: switch.sw_tester_cl

Customized them a bit for eye candy and functionality thanks to @andrey’s awesome customize-ui and here goes:

29
29782×498 88.8 KB

to the left the now truncated Command_line switches with the get state taken out, hence the bolts instead of the toggle. To the right the new template switches, working properly as they are.

My logfile has calmed down like you can’t imagine! :+1:

no more state polling every 15 seconds, and the @mviezzer 's adjusted summary.py has calmed down also :wink:

only thing bothering still is the Hassio specs growing out of hand even further…

15

anyways, thank you very much!
Marius

52

Very cool. You can hide the curl switches and binary_sensor in customize.yaml to get them out of your way.

For MariaDB, you can control size by setting purge_interval and purge_keep_days in the recorder component. You can also exclude any entities that you don’t care about tracking.

Personally, I use InfluxDB and Grafana for graphing historical data and limit MariaDB to 1 day because the MariaDB database gets huge fast.

To figure out what’s using all that disk space, you can ssh into the host and look around by following steps shown at Debugging the Home Assistant Supervisor | Home Assistant Developer Docs. The place you want to check out is /resin-data.

53

quite! it took me all afternoon but now im quite happy with the result. While i was at it, i created a few very simple threshold template_binaries also, just incase :wink:

52
52376×608 31.5 KB
01
41
41388×664 43.3 KB

i have yet to find my may in deleting sensors and switches i have created carefully in the past. For now, commenting them out and hiding in customize suffices. When im satisfied all is well and i don’t need the older definitions anymore, ill purge my config files somewhat further.

On the DB:

the thing is, i started this whole excersize because i want historical data to take me at least one year back, to be able to compare energy usage et all. Purging the DB after 1 day doesn’t help then…

I was led to believe using a real DB like Maria DB should allow me to do so?
Do you say you use both InfluxDb and Grafana ánd MariaDB?
DO Influx and Grafana take care of the longer period im looking for? If so, how do i set that up, i really really need that priority no.1 right now. Hope you care to share your setup on that.

My HA comes to a stand still and runs out of memory when i click history or logbook in the settings. Something must be amiss, and there really should be some kind of safety or security settings to prevent that.

Im still searching for a way to write the DB to an attached usb stick, this should be a possibilty, since when i browsed my flashcard I bumped into the Media folder containing usb, floppy and cdrom…

43
ive yet to find a way to point HA to those folder though.

Cheers!
Marius

1 Like
54

In theory it can, but not sure a PI can handle that much data.

Yes, I use InfluxDB for historical data. I use MariaDB for state restoration of input_* devices and to give me a little bit of history natively in the Home Assistant UI.

For my system, the data directory for MariaDB with 1 day of data is 374 MB and the data directory for InfluxDB with 5 months of data in InfluxDB is 534 MB.

If you want to try InfluxDB and Grafana, check out Repository: Bestlibre Addons repository.

That’s likely because your MariaDB database is so large that the PI is struggling to handle it. If you don’t want to reduce the amount of data, you might want to move to more powerful hardware such as an old PC.

Before I moved off of a PI, I used USB Boot on Raspberry Pi 3 - #37 by xbmcnut to run hassio on a USB attached SSD. When I was using it, it performed a lot better than running from an sdcard. In the end, I just have way too much stuff for a PI to handle. I now run hassio on a VM, and the performance improvement is significant.

55

where is this on a Hassio install?
the only data folder i can find is rather empty., and only contains ssh keys.

i am logged in as root, which is the only way i know how to ssh in to my hassio.local, and also to ftp using cyberduck, showing all files on the sd card (i think)

btw, my home-assistant_v2.db file is so small (113mb) because after i installed the MariaDb it never changed :wink: can’t be the troublemaker.
this is my db_url: mysql://login:password@core-mariadb/homeassistant. Now where is that ?

Cheers,
Marius

56

Follow instructions at Debugging the Home Assistant Supervisor | Home Assistant Developer Docs to be able to ssh to the host. Once you do that, you should be able to see /resin-data in that ssh session.

57

My Pi B+ has been running for over a year with influxdb, so a Pi3 should handle it comfortably. I run grafana on my PC to take the graphics workload off the pi.

58

thanks.
what is your config for influxdb, how did you set that up? I am on Hassio and installing outside of the Hassio add-on system is somewhat complex now and then…

also, why would be the advantage of Influx over MariaDB, since the latter is offered as an add-on, i would have thought that to be the first choice of the dev’s?

Cheers,
Marius

59

HI @NotoriousBDG,

please let me get back on this for a bit. Since ive made these changes ive run into a new sort of trouble… my binary state sensors don’t get updated very often of course, since they’re always on. somehow, this is not always seen, and all of my sensors show off, while my threshold_sensors, that measure the real power usage show they’re on, because they are using power.
Since the binary state sensors are used in the rest of my system, and automations, this causes big trouble, which not even a host reboot solves all the time.

Could this have to do with the recorder bit of HA? I’ve also tried to reduce the history/logging of HA shortly after i had changed the template switches/sensors you helped me with.

If so, would you know how to be sure these template sensors stay populated? I should not be an issue of the mqtt events, since they are published with the retain value. I do have qos set to 1, maybe i should change that?

just for completeness here’s the template:

sw_tester_template:
  friendly_name: "7 - Tester"
  value_template: "{{ is_state('binary_sensor.tester_state_bin', 'on') }}"
  turn_on:
    service: switch.turn_on
    data:
      entity_id: switch.sw_tester_cl
  turn_off:
    service: switch.turn_off
    data:
      entity_id: switch.sw_tester_cl

and the binary sensor:

- platform: mqtt
  name: "Tester state bin"
  state_topic: 'mac_address/powerswitch-zwave/4e6eeb01/state'
  payload_on: "on"
  payload_off: "off"
#  availability_topic: "home-assistant/window/availability"
#  payload_available: "online"
#  payload_not_available: "offline"
  qos: 1
  device_class: plug
#  value_template: '{{ value.x }}'

the #-commented lines are not mine, their were in the example code I that don’t fit in my setup

the command_line switch

sw_tester_cl:
  friendly_name: Tester (cl)
  command_on: >-
    curl -X POST -d '{"seq":1, "method":"object_prop_set", "arguments":{"oid":"4e6eeb01", "prop":"command", "value":"on"}}' http://192.168.xxx.xxx/iungo/api_request
  command_off: >-
    curl -X POST -d '{"seq":1, "method":"object_prop_set", "arguments":{"oid":"4e6eeb01", "prop":"command", "value":"off"}}' http://192.168.xxx.xxx/iungo/api_request#      command_state: >-
#      command_state: >-
#        curl -X POST -d '{"seq":1, "method":"object_prop_get", "arguments":{"oid":"4e6eeb01", "prop":"state"}}' http://192.168.xxx.xxx/iungo/api_request
  value_template: >-
    {%- if value_json.rv.value == 'on' -%}
    {{ true }}
    {%- else -%}
    {{ false }}
    {%- endif -%}

Please have a look if you would,
Thanks,
Marius

60

Influxdb is a time series database, designed to store series of data, and so is much more efficient at it. MariaDB is a relational database, which is good at storing complex relationships, but at the cost of efficiency for relatively simple tasks like this.

I have not used Hassio, but I would be surprised if there isn’t an addon for Influxdb.

61

alright, thx.
and you have set excludes/includes for recorder, history and logbook? They wont load in my setting, and overload the system to a standstill.
If so, please would you share you’re settings there? Since you run on a Pi, that would be good reference for me!

Ive set includes /excludes for now, but am not very confident ive set them tight enough.

As i understand it those are than read by MariaDB, and the Influx would create its own db, read by grafana on another device?

62

Just to make clear, what is stored in the influxdb database is configured in the Influxdb component settings, and is completely independent of the history and recorder components.

I use the default sqlite rather than Mariadb and my settings are quite simple

# Enables support for tracking state changes over time.
history:

# The recorder component is storing details in a database which then are handled by the history componen
recorder:
  purge_interval: 1
  purge_keep_days: 3
  exclude:
    domains:
      - weblink
      - updater

The sqllite database file is currently 36M.

For influxdb I specify the entities I want to store

influxdb:
  host: 127.0.0.1
  username: home-assistant
  password:  !secret influxdb
  include:
    entities:
      - binary_sensor.lounge_motion_sensor
      +49 others
1 Like
63

thx!
I will check that out asap

64

Are you sure the retain flag is set? Your symptoms suggest that it isn’t.

65

There are addons available for both InfluxDB and Grafana at Repository: Bestlibre Addons repository. Those are the ones that I use.

66

yes! all other properties are retained… Ive just mailed the devs of the Iungo if this is expected behavior and if they could see on/off to be published once a minute or so.

What makes it even stranger is that is starts-off alright, and after some time suddenly the sensors are dead.

Ive even suspected my editing several automations to be the culprit, but is has roven to be the other way round. No on/off state read or resent.
Maybe my setting the recorder and logger/history options could have interfered? Ive included switches, sensors and binary_sensors, but still things went wrong this morning.

Did a reboot of the Iungo hub, and all sensors were populated again, thank goodness. Proves the code in HA is alright? or should anything here be changed:

- platform: mqtt
  name: "Tester state bin"
  state_topic: 'mac_address/powerswitch-zwave/4e6eeb01/state'
  payload_on: "on"
  payload_off: "off"
#  availability_topic: "home-assistant/window/availability"
#  payload_available: "online"
#  payload_not_available: "offline"
  qos: 1
  device_class: plug

have even changed the qos to 0, but that didnt help. wanted it to 1 just because of this behavior, what do you think?

Thanks,
Marius

67

ive checked that out, but it still is a bit off of a add-on… lots of finetuning still necessary. Not afraid of that, but i would love to be able to use this, to write the DB to an attached thumb.
Id probably
1- have to instruct Hassio to mount and use the thumb and then
2- instruct InfluxDB to use the thumb

using this code that seems to be an option:

This section controls where the actual shard data for InfluxDB lives and how it is flushed from the WAL. dir may need to be changed to a suitable place for you system, but the WAL settings are an advanced configuration. The defaults should work for most systems.
dir = “/var/lib/influxdb/data” The directory where InfluxDB stores the data. This directory may be changed. Note: The default directory for OS X installations is /Users/<username>/.influxdb/data Environment variable: INFLUXDB_DATA_DIR

Above is copied from their website Configure InfluxDB OSS | InfluxDB OSS v1 Documentation

So i guess what im asking is is the above correct, and if yes, whether you can help me formulate a way to implement 1 and 2 :wink:

maybe this should go to a separate thread?

Marius

68

Setting the retain flag would be better than sending status every minute. The state would always be correct even when Home Assistant restarts.

Not sure what you mean here. Try following the mqtt topic to see what was sent at the time the problem happens.

The mqtt binary_sensor doesn’t restore state from the recorder component. To have the correct state on startup, the retain flag must be set.