Unreliable InfluxDB size sensor

That’s kinda sad… sleep is not supposed to do that. Mine did just like yours when I tried the read -t 10 command as the pause (verified with htop in the shell). Sleep does not seem to do what it’s doing to you… works like a charm for me.

I just tried it again. This time it seems to be behaving. Not sure what I did wrong last time.

Will post up the full solution after testing for a few days.

Interesting :+1:t3:

Update after 3 days? Worth trying or still side effects (performance/resource consumption, non-working SSH addon, …)?

No performance issues but the value has not changed since it started. I need to investigate why when I get some time, later today or tomorrow.

Ah okay. Keep us posted :+1:t3:

1 Like

If you’re using Linux and InfluxDB addon in HASSIO make some_file.sh with content:

#!/bin/bash


mqttuser='your_user'
mqttpassword='your_password'
mqttbroker='your_broker_ip_or_localhost'
mqttport='your_port'

influxdb_size=$(docker exec YOUR_INFLUXDB_DOCKER du -shm "/data/influxdb/data/homeassistant" | cut -f1 )
mosquitto_pub -h $mqttbroker -p $mqttport -u $mqttuser -P $mqttpassword -t influxdb/influxdb_size -m $influxdb_size -r

This file will output size of folder in MB and publish it in mqtt: influxdb/influxdb_size

Next create the mqtt sensor:

  - platform: mqtt
    name: influxdb_size
    state_topic: "influxdb/influxdb_size"
    unit_of_measurement: "MB"

And in the end add to cron file ‘some_file.sh’ with interval whatever you want.
Hope it will help you

Thanks but I’m using HassOS. So that is not possible. My workaround using the SSH web terminal is working.

1 Like

Using HASS OS too. And not using MQTT.

So what about

?

Any smaller issues, side effects or inconveniences? Or “bullet proof” to use?

All going swimmingly.

So, to wrap it all up. Use this init_command in the SSH & Web Terminal addon with protection mode disabled:

init_commands:
  - >-
    while [ 1 = 1 ]; do docker exec addon_a0d7b954_influxdb du -s
    /data/influxdb/data/homeassistant >
    /config/file_sensors/ha_influx_db_size.txt && sleep 300; done &

Then create this file sensor:

- platform: file
  name: 'InfluxDB HA DB Size'
  file_path: '/config/file_sensors/ha_influx_db_size.txt'
  unit_of_measurement: 'MB'
  value_template: "{{ (value.split('\t')[0]|int/1000)|round(3) }}"

You can then disable the InfluxDB internal database if you wish (recommended by InfluxDB).

To do so, in the Influxdb addon configuration add this:

envvars:
  - name: INFLUXDB_MONITOR_STORE_ENABLED
    value: "false"

Don’t expect it to be as active as the shard method. I only see an increase every day or two.

2 Likes

Very cool, you got it done! The workaround being leveraged is obviously overkill :slight_smile: but you gotta do what you gotta do in a restricted environment… Glad some of my quick attempts actually got this moving forward for you @tom_l.

Yeah it’s a bit like cracking a walnut with a sledgehammer.

Thanks for your assistance. Much appreciated.

1 Like

I am seeing this error occasionally:

Logger: homeassistant.components.file.sensor
Source: components/file/sensor.py:85
Integration: file (documentation, issues)
First occurred: 17 February 2021, 13:42:25 (2 occurrences)
Last logged: 17 February 2021, 13:57:26

File or data not present at the moment: ha_influx_db_size.txt

Possibly because the file is being written to when home assistant is trying to access it.

As an alternative I’m looking to push the update directly from the command using the Home Assistant rest API, see this example:

curl -X POST -H "Authorization: Bearer ABCDEFGH" \
  -H "Content-Type: application/json" \
  -d '{"state": "25", "attributes": {"unit_of_measurement": "°C"}}' \
  http://localhost:8123/api/states/sensor.kitchen_temperature

I finally decided to

  1. create my InfluxDB database sensors by
  2. relying on the folder size instead of the _internal DBs shard series field “diskBytes” as it is hilarious small compared to the actual folder size (approx. 56 times smaller) and therefore
  3. to use a more basic approach by using a simple command_line sensor. That one looks like
  - platform: command_line
    name: InfluxDB size (homeassistant)
    scan_interval: 120 # Change to higher value after finishing tests
    command_timeout: 30
    command: "docker exec addon_XXXXXXXX_influxdb du -shm /data/influxdb/data/homeassistant | cut -f1"
    unit_of_measurement: MB
    value_template: "{{ value }}"

I reloaded command line entities (did not restart HA) and it has been created.
BUT: it shows a size of zero (0 MB)

grafik

The command itself is working perfectly when fired on the SSH console. What did I miss?

Update 1:
I´m pretty sure I found the root cause thinking bout it for 10 seconds. The command is performed from homeassistant so from inside the homeassistant container (I´m running HASS OS too). And there even is no docker command inside “homeassistant”. So any way to run that command for the command line sensor with “root privileges” (how to break out of the HA docker container?)? I´m pretty sure that´s impossible.

Running it from root ssh, storing it in a text file and reading that one from homeassistant as sensor now sounds pretty cool to me (not that much overkill)… would still like to avoid this, hopefully there´s a smarter solution.

Man I liked to keep it simple so much… what is the next approach without pushing it as far as @tom_l :slight_smile: ?

Update 2:
My last approach was a “hammer style” one: using SSH to login to the host and get the information. Unfortunately this comes with several detailled and overall security risks like

  1. by either passing the SSH password for the HA host on the command line sensor or
  2. the need to whitelist the HA container in the host known_hosts file (password-less SSH),
  3. in the end the biggest security issue would be HA container being able to access the host - which is a nice “Welcome! Get in!” for every bad guy, especially when having HA publicly accessable on the internet…
    In my opinition homeassistant container should not have the opportunity to get access to the host. So providing the information from the host for the homeassistant container is also complicated but quite good in terms of security.

Not sure how to proceed… :woozy_face:

1 Like

i tried to make the sensor same way you described. but i got no sensor and a error msg.

Logger: homeassistant.components.file.sensor
Source: components/file/sensor.py:45
Integration: file (documentation, issues)
First occurred: October 27, 2021, 17:03:22 (1 occurrences)
Last logged: October 27, 2021, 17:03:22

'/config/file_sensors/ha_influx_db_size.txt' is not an allowed directory

You need to allow files to be read from there: https://www.home-assistant.io/docs/configuration/basic/#allowlist_external_dirs

configuration.yaml

homeassistant:
  allowlist_external_dirs:
    - /config/file_sensors

Hi

What about using a MQTT sensor in stead of using a file?!

Command (pipe to MQTT in stead to a file):

init_commands:
  - >-
     docker exec addon_a0d7b954_influxdb du -s /data/influxdb/data/homeassistant | 
     mosquitto_pub -t sensors/influxdb/dbsize -u <user> -P <password> -l  && sleep 300; done &

Sensor config:

name: InfluxDB size
unit_of_measurement: 'MB'
icon: hass:chart-line
state_topic: "sensors/influxdb/dbsize"
value_template: "{{ (value.split('\t')[0]|int(0)/1000)|round(3) }}" 
5 Likes

That’s an excellent idea Eric! Works perfectly. Hopefully it will solve this issue too:

The only addition I’ve made is to add the retained flag so that the sensor restores after a Home Assistant restart and to change the topic to match the structure I use:

init_commands:
  - >-
    while [ 1 = 1 ]; do docker exec addon_a0d7b954_influxdb du -s
    /data/influxdb/data/homeassistant | mosquitto_pub -t
    home-assistant/sensor/dbsize -r -u <user> -P <password> -l && sleep
    300; done &

Mqtt sensor:

- name: InfluxDB DB Size
  unit_of_measurement: 'MB'
  icon: hass:chart-line
  state_topic: "home-assistant/sensor/dbsize"
  value_template: "{{ (value.split('\t')[0]|int(0)/1000)|round(3) }}" 
4 Likes

Hi, may I please have some advise. I have don all of the above and get the following:

My config in the addon seems correct and I can use the user creds to login to homeassistant…

Those credentials are for your mqtt broker.