Grafana display problems

Hi there,

my name is Charly and I’m quite happy with HA. I started beginning of this year with getting my Sonoff’s back from China and into HA. I’m using Tasmota and every things works very good so far.

My only problem is, that some temperatures monitored with Influx / Grafana are not displayed properly. Even if the values are submitted by HA very well and 24/7.

I assume, a picture says more:

(since I’m a new user, only picture is allowed… ;-))

So, for me it looks like that the data is not getting into Influx / Grafana…

Would be very nice, if someone is having a hint for me…

Thanks a lot in advance,
charly

Perhaps what we discussed here the other day here:

HA stores data only on state change.
The line in your graph from HA covers 24 hours, but it seams to be a flat line at certain times, after 06:00am and perhaps again around 12 noon.

Can you post your query in Grafana? is it like

SELECT mean("value") FROM "temperature" WHERE ("entity_id" = 'bme680_temperature_db') AND time >= now() - 30m GROUP BY time(1s) fill(previous)

Grafana is looking for values stored within the past 30 minutes, but the line in the graph stars with the first timestamp found within the timeframe, “fill(previous)” fills the gaps after the first value found.

You can check the data in the measurement with “Explore” from Grafana or using the influx-cli
in my system for example:

As described in the other thread my workaround is a template sensor based on the actual sensor, with an additional attribute (“update_now”) based on the time, attribute changes every 5 minutes and thus forces a value to be stored in influx. Of course still have gaps on the left side for a maximum of 5 minutes, but that’s ok for me and limits the number of values written to the database.

Armin

1 Like

Hi Armin,

thank you for your quick reply…

I wish I could send you a query, but I don’t know how to get to the command line… :flushed:…sorry, I’m a newbie…
My HA is running on a Raspi and I’m accessing it using the browser…

charly

Hi

I would suggest to create a new panel in a testing dashboard in Grafana, with only a single query, makes it easier to analyse.

from Grafana you can click the “down”-arrow next to titel of your panel and select “Edit”, or press “e” on the keyboard while your mouse is over the graph.
Then below the graph on right side you should see a button “Query Inspector”. After clicking “Query Inspector” you should see something like this:

This is the query being used by Grafana, click on “data” to show the data returned from influx
Grafana builds the query based on the timeframe you want to see ( for example 6 hours means “time >= now() - 6h” ) and how big one step is horizontal ( here a datapoint every 10 seconds)

In my data you can see that it starts at 09:15:10, but the first non-null value is 09:18:00

in the graph this means that the line begins at 09:18:00, timeline starts at 09:15:10 => no graph for 3 minutes on the left side
The influx integration in HA stores only when a value changes, your gap between two data points is like much longer, like I said I fixed this for me by adding template sensors and changing one attribute of the new sensor every 5 minutes. But this requires editing in the yaml-configuration files
5 minutes is just good enough for my system, I just ignore that depending on time resolution I have gaps for up to 5 minutes at the beginning of the line.

Armin

Hi Armin,

thanks again for this explanation. I hope, I understand everything correctly :wink: So, Influx stores only data, if a value is changing. So it means, no temperature change = no value in influx DB = no value in Grafana.

So, I’m having some sensor that deliver data 24 hours and some not. I like to try out different sensor, like DS18B20, AM3201, BMP280 could this maybe a reason? Most of my devices are running on Tasmota.

If it is not too much, can you write me please, how and where to change the value in the yaml configuration files? I understood, that there will be a gap of 5 mins at the start of putting in a new sensor or query, which is of course okay for me…

Vielen Dank…
charly

Hi.
I try to explain how I did it, I’m using HA for about 4 weeks now, currently migrating from Openhab,
But I have more than 25 years experience in Linux, makes it easier for me to tweak files, restart services, read the logs, …

I don’t know the sensors you are using, hope it works with them as it does with mine. What I do have is thermostats for Bosch SmartHomeControl and one Bosch BME680 Sensor directly attached to one of my Raspis, the Bosch SHC has integration in HA, from the BME680 I retrieve data using a python script and mqtt.

Am I right that you are using Home Assistant OS (image from HA flashed to the SD card)? I’m using Home Assistant Core ( manual installation of all HA components on a standard Raspi OS).
In HA OS you might need the File Editor(?) or Samba Plugins to get direct access to the config files

Also please make copies of any files you change, python and yaml is very picky about the formatting and indentation, one little error and it doesn’t like the file anymore. For Editing from a Windows System I strongly recommend tools like notepad++ or Visual Studio Code.

As already explain I’m using template sensors based on a recommendation from other community members. A template sensor is, in simple words, a manually created sensor retrieving its values from another entity. With the template you can how to create the value ( calculation, formatting, …) and you can also add addtional attributes.
I’m not aware that a template sensor can be created with the Web-GUI, thus tweaking the files is required.

Locate the configuration folder in your system, this is the folder containing “configuration.yaml”
Create a new file “template_sensor.yaml”
Define the first template, this is a sample

heizungwz1_temperature_db:
  value_template: "{{ states('sensor.heizungwz1_temperature') }}"
  friendly_name: "Heizung Wohnzimmer 1 Temperatur"
  unique_id: heizungwz1_temperature_db
  device_class: temperature
  attribute_templates:
    update_now: "{{ (now().minute / 5) | round(0) }}"

heizungwz1_temperature_db: this is the name for the new entity. All my entities going to Influx are the original entity_id ( without “sensor.” ) suffixed with “_db”, I’ll get back to that with the influx configuration

value_template: the name in quotes refers to the entity_id of the original sensors, the sensor our new template is based on

unique_id: not sure if I really require this, but it didn’t hurt so far…

device_class: based on list here, tells HA what kind of sensors this is… https://www.home-assistant.io/integrations/sensor/

attribute_templates: the addtional attribute for my sensor, forcing it to update

update_now: name of the attribute, in quotes the calculation to update it every 5 minutes. You can also use “{{ (now()}}” to update every minute. But that’s 1440 rows for each sensor per day, I decided that 5 minutes is good enough for me, and least when storing something related to room temperature

The template language is very powerfull, you can do a lot more than what I’m doing here

Next step is to include to new file in the main configuration file “configuration.yaml”. Open the file and check if you already have a line starting “sensor:”, if not add it at the end of the file. Then include “template_sensor.yaml” for the platform “template” (my file has additional sensors for mqtt and my Fritzbox), the new entry is the last in the list:

sensor:
  - platform: mqtt
    name: "BME680 Temperature"
    state_topic: "sensors/bme680"
    unit_of_measurement: '°C'
    value_template: "{{ value_json.Temperature }}"
    unique_id: bme680_temperature
  - platform: fritzbox_netmonitor
    host: 192.168.0.1
    name: "Fritz DSL"
  - platform: template
    sensors: !include template_sensor.yaml

if you have no other sensors already defined, it looks like:

sensor:
  - platform: template
    sensors: !include template_sensor.yaml

After restarting you should see a new entity named “sensor.heizungwz1_temperature_db” ( or any other name you choose above)

My influxdb integration is this:

influxdb:
  host: 127.0.0.1
  port: 8086
  database: homeassistant
  username: someuser
  password: somepassword
  default_measurement: state
  include:
    entity_globs:
      - binary_sensor.*_db
      - sensor.*_db
  component_config_glob:
    sensor.*humidity*:
      override_measurement: humidity
    sensor.*temperature*:
      override_measurement: temperature  
    sensor.*valve*:
      override_measurement: valve
    sensor.*battery*:
      override_measurement: battery
    sensor.*light*:
      override_measurement: light
    sensor.*synology*:
      override_measurement: synology

the entity_globs in “include” defines the entities to store in influx, I choose to store only entities with suffix “*_db”, gives me much finer control on what is going to influx.
In component_config_glob I’m overwriting the names of the measurements used in influx, the influx integration in HA has the odd habit to use the unit of the sensor as name, you then have measurements like “°C” or “%”… I just didn’t like the naming… With the override I for example define that entities matching “sensor.temperature” are going to “temperature” in influx.

sorry, that’s a lot to read, hope it helps.
If somebody from the community knows an easier way, just respond here.

Armin

1 Like

Hi
missed one thing… when all is done and your sensors are updated in influx for grafana, you might want to add this to “configuration.yaml”

recorder:
  exclude:
    entity_globs:
      - binary_sensor.*_db
      - sensor.*_db

this stops the sensors created for influx from being stored in the history database in HA, no duplicate data
Armin

Hi Armin,

thank you for your very detailed explanation. Since I’m pretty busy with work at the moment and not at home, I cannot try to implete this into my HA.
But when Im back, I will try it and let you know…

And yes, IF somebody knows how the get this easier, I would also appreciate it very much… :wink:

Thanks again,
charly

Hi
perhaps this helps as a frist step, after enabling SMB you should be able to access the config folder from a Windows PC.
https://www.byteshark.de/index.php/computer/smarthome/311-home-assistant-samba

It’s in german, but based on your alias here and the screenshots you provided earlier, I assume german is your primary language anyway.

Armin