Unreliable InfluxDB size sensor

I am seeing this error occasionally:

Logger: homeassistant.components.file.sensor
Source: components/file/sensor.py:85
Integration: file (documentation, issues)
First occurred: 17 February 2021, 13:42:25 (2 occurrences)
Last logged: 17 February 2021, 13:57:26

File or data not present at the moment: ha_influx_db_size.txt

Possibly because the file is being written to when home assistant is trying to access it.

As an alternative I’m looking to push the update directly from the command using the Home Assistant rest API, see this example:

curl -X POST -H "Authorization: Bearer ABCDEFGH" \
  -H "Content-Type: application/json" \
  -d '{"state": "25", "attributes": {"unit_of_measurement": "°C"}}' \
  http://localhost:8123/api/states/sensor.kitchen_temperature

I finally decided to

  1. create my InfluxDB database sensors by
  2. relying on the folder size instead of the _internal DBs shard series field “diskBytes” as it is hilarious small compared to the actual folder size (approx. 56 times smaller) and therefore
  3. to use a more basic approach by using a simple command_line sensor. That one looks like
  - platform: command_line
    name: InfluxDB size (homeassistant)
    scan_interval: 120 # Change to higher value after finishing tests
    command_timeout: 30
    command: "docker exec addon_XXXXXXXX_influxdb du -shm /data/influxdb/data/homeassistant | cut -f1"
    unit_of_measurement: MB
    value_template: "{{ value }}"

I reloaded command line entities (did not restart HA) and it has been created.
BUT: it shows a size of zero (0 MB)

grafik

The command itself is working perfectly when fired on the SSH console. What did I miss?

Update 1:
I´m pretty sure I found the root cause thinking bout it for 10 seconds. The command is performed from homeassistant so from inside the homeassistant container (I´m running HASS OS too). And there even is no docker command inside “homeassistant”. So any way to run that command for the command line sensor with “root privileges” (how to break out of the HA docker container?)? I´m pretty sure that´s impossible.

Running it from root ssh, storing it in a text file and reading that one from homeassistant as sensor now sounds pretty cool to me (not that much overkill)… would still like to avoid this, hopefully there´s a smarter solution.

Man I liked to keep it simple so much… what is the next approach without pushing it as far as @tom_l :slight_smile: ?

Update 2:
My last approach was a “hammer style” one: using SSH to login to the host and get the information. Unfortunately this comes with several detailled and overall security risks like

  1. by either passing the SSH password for the HA host on the command line sensor or
  2. the need to whitelist the HA container in the host known_hosts file (password-less SSH),
  3. in the end the biggest security issue would be HA container being able to access the host - which is a nice “Welcome! Get in!” for every bad guy, especially when having HA publicly accessable on the internet…
    In my opinition homeassistant container should not have the opportunity to get access to the host. So providing the information from the host for the homeassistant container is also complicated but quite good in terms of security.

Not sure how to proceed… :woozy_face:

1 Like

i tried to make the sensor same way you described. but i got no sensor and a error msg.

Logger: homeassistant.components.file.sensor
Source: components/file/sensor.py:45
Integration: file (documentation, issues)
First occurred: October 27, 2021, 17:03:22 (1 occurrences)
Last logged: October 27, 2021, 17:03:22

'/config/file_sensors/ha_influx_db_size.txt' is not an allowed directory

You need to allow files to be read from there: https://www.home-assistant.io/docs/configuration/basic/#allowlist_external_dirs

configuration.yaml

homeassistant:
  allowlist_external_dirs:
    - /config/file_sensors

Hi

What about using a MQTT sensor in stead of using a file?!

Command (pipe to MQTT in stead to a file):

init_commands:
  - >-
     docker exec addon_a0d7b954_influxdb du -s /data/influxdb/data/homeassistant | 
     mosquitto_pub -t sensors/influxdb/dbsize -u <user> -P <password> -l  && sleep 300; done &

Sensor config:

name: InfluxDB size
unit_of_measurement: 'MB'
icon: hass:chart-line
state_topic: "sensors/influxdb/dbsize"
value_template: "{{ (value.split('\t')[0]|int(0)/1000)|round(3) }}" 
5 Likes

That’s an excellent idea Eric! Works perfectly. Hopefully it will solve this issue too:

The only addition I’ve made is to add the retained flag so that the sensor restores after a Home Assistant restart and to change the topic to match the structure I use:

init_commands:
  - >-
    while [ 1 = 1 ]; do docker exec addon_a0d7b954_influxdb du -s
    /data/influxdb/data/homeassistant | mosquitto_pub -t
    home-assistant/sensor/dbsize -r -u <user> -P <password> -l && sleep
    300; done &

Mqtt sensor:

- name: InfluxDB DB Size
  unit_of_measurement: 'MB'
  icon: hass:chart-line
  state_topic: "home-assistant/sensor/dbsize"
  value_template: "{{ (value.split('\t')[0]|int(0)/1000)|round(3) }}" 
4 Likes

Hi, may I please have some advise. I have don all of the above and get the following:

My config in the addon seems correct and I can use the user creds to login to homeassistant…

Those credentials are for your mqtt broker.

I use the addon broker and that user homeassistant users, right?

No you can’t use the homeassistant user. Use the credentials your devices use to connect to the broker.

Sorry, I am not sure if I understand… I use a home assistant user to connect to mosquitto.


Do I need to make like above? Why?

And in the addon docs:
Option: logins (optional)
A list of local users that will be created with username and password. You don’t need to do this because you can use Home Assistant users too, without any configuration. If a local user is specifically desired:

logins:

  • username: user
    password: passwd

What am I mssing?

Yes you can but you specifically can not use the username “homeassistant”.

https://github.com/home-assistant/addons/blob/5d21f08245dd0fac4183169fe43848c05547c401/mosquitto/DOCS.md

Thanks for the retain flag suggestion (adding -r flag)

I did not use that username…

Hi!
I can’t make the Terminal&SSH work. I have tried the file creation mode and the mqtt, but it seems, that nothing happened.
I don’t have an option to turn off Protection mode

config:

log

Thank you for your help!

Turn on advanced mode in your profile.

That add-on won’t work. In the documentation it says

Known issues and limitations

  • This add-on will not enable you to install packages or do anything as root. This is not working with Home Assistant.

Use the SSH & Web Terminal add-on found here GitHub - hassio-addons/addon-ssh: SSH & Web Terminal - Home Assistant Community Add-ons

BTW, @tom_l and @erkr thank you for the tips for running the MQTT publish as part of the ssh add-on, works great!
image

1 Like

Doh. Good catch. I missed that they were not using the Web terminal SSH add-on version.

Hi,
Previously in this thread I proposed to publish the InfluxDB size via MQTT. I just want to update how to publish it via a command line sensor, for those that don’t want to run a MQTT server. Please note this procedure is more advanced.

  • The SSH & Web Terminal is still needed (with protection mode off)
  • We now don’t create the init command in that add-on!
  • You first need to create authorization keys to enable password less SSH access. This is explained in: SSH'ing from a command line sensor or shell command
  • the procedure doesn’t mention you have to also have add the public key to the authorization_keys in the SSH & Web terminal config!
  • If SSH logon works without a password, you can create a command line sensor to read the InfluxDB size like this:
  - platform: command_line
    name: InfluxDB Size
    unique_id: influxdb_size
    command: ssh <user>@192.168.178.47 -i /config/.ssh/id_rsa 'sudo docker exec addon_a0d7b954_influxdb du -s /data/influxdb/data/homeassistant'
    value_template: "{{ (value.split('\t')[0]|int(0)/1000)|round(1) }}" 
    unit_of_measurement: 'MB'
    scan_interval: 300  

Check the log of your SSH & Web terminal add-on if there are issues

2 Likes

Each addon has a hostname which is shown on the info tab of the config panel for that addon which can be used to talk to it from HA and other addons. You don’t have to expose the ssh on the host and go out over your LAN to talk to it from HA. Just use this instead and talk to it directly without leaving the docker network:

ssh <user>@a0d7b954-ssh ...

Plus then your yaml can simply be copied and pasted by other users that want to do the same since the hostname of an addon is the same on all systems. Whereas everyone’s LAN subnet and chosen IP for HA is different.

3 Likes