Data from ESPHome with Modbus is not recorded or at least not correctly shown in Grafana

I have a problem with data recording. I’m not sure if it’s a Home Assistant problem, one from Grafana or one from ESPhome.
Initial situation:
I query two SDM72 via -platform: sdm_meter. On one, all available parameters are recorded. On the other one (for test purposes - to keep the query time as short as possible), only the voltage and current of L1.
Error description:
Very few measured values are displayed in Grafana. For test purposes, I record a value every three seconds. All measured values are displayed in the ESPhome logging console. It is also indicated that they are transmitted to Home Assistant.
Large gaps are visible in Grafana.
These gaps are not present in Home Assistant (History). However, I do not know whether they have really been recorded/transferred/saved or whether Home Assistant is “cleverly” interpolating.
My own ideas and observations:
If the values change regularly (as with the voltage, which is transmitted with two decimal places), continuous graphs appear in Grafana. If they have a constant value for a longer period of time (such as the current when the instantaneous water heater is not used, i.e. it is zero) then gaps appear in Grafana.
Of course, I can use tricks to fake the values (fill “zero” in the query or “Step After” in the line interpolation). But that is not how it should be.
I would first like to check exactly what data Home Assistant receives and where it stores the data.


Log of ESPHome - Not the same Time Period but it results in the same Effect.

ESPHome Configuration
packages:
  common: !include .common.yaml

esphome:
  name: "esphome-c3-modbus"
  friendly_name: "ESPHome Modbus"
  min_version: 2024.6.0
  name_add_mac_suffix: false
  project:
    name: esphome.web
    version: dev

esp32:
  board: esp32-c3-devkitm-1
  variant: ESP32C3
  framework:
    type: esp-idf

i2c:
  sda: GPIO4
  scl: GPIO3

# Example configuration entry with 2 sensors and filter

# wireguard:
#   address: 192.168.14.129
#   private_key: !secret wg_private_key
#   peer_endpoint: !secret wg_endpoint
#   peer_public_key: HDOFIvQkrU39eDT6uTANbcbqs8JGp17l+RUutBc4xho=

#   # Optional netmask (this is the default, no outgoing traffic
#   # will pass through the tunnel if omitted)
#   netmask: 255.255.255.0

#   # Optional endpoint port (WireGuard default if omitted)
#   peer_port: 58518

#   # Optional pre-shared key (omit if not in use)
#   peer_preshared_key: !secret wg_shared_key

#   # Optional list of ip/mask (any host is allowed if omitted)
#   peer_allowed_ips:
#     - 192.168.14.0/24
#     - 0.0.0.0/0
#   require_connection_to_proceed: True
  
#   # Optional keepalive (disabled by default)
#   peer_persistent_keepalive: 25s  
  
# binary_sensor:
# - platform: wireguard
#   status:
#     name: 'WireGuard HIG Status'
# text_sensor:
#   - platform: wireguard
#     address:
#       name: 'WireGuard Address'



uart:
  id: uart_modbus
  tx_pin: GPIO21
  rx_pin: GPIO20
  baud_rate: 9600
  data_bits: 8
  stop_bits: 1
  parity: NONE

modbus:
  flow_control_pin: GPIO0
  id: modbus1

sensor:
  - platform: wifi_signal # Reports the WiFi signal strength/RSSI in dB
    name: "WiFi Signal dB"
    id: wifi_signal_db
    update_interval: 60s
    entity_category: "diagnostic"

  - platform: copy # Reports the WiFi signal strength in %
    source_id: wifi_signal_db
    name: "WiFi Signal Percent"
    filters:
      - lambda: return min(max(2 * (x + 100.0), 0.0), 100.0);
    unit_of_measurement: "Signal %"
    entity_category: "diagnostic"
    device_class: ""

  - platform: sdm_meter
    address: 10
    phase_a:
      current:
        name: "Durchlauferhitzer L1 Strom"
      voltage:
        name: "Durchlauferhitzer L1 Spannung"
      active_power:
        name: "Durchlauferhitzer L1 Wirkleistung"
      power_factor:
        name: "Durchlauferhitzer L1 Power Factor"
      apparent_power:
        name: "Durchlauferhitzer L1 Scheinleistung"
      reactive_power:
        name: "Durchlauferhitzer L1 Blindleistung"
      phase_angle:
        name: "Durchlauferhitzer L1 Phasenwinkel"
    phase_b:
      current:
        name: "Durchlauferhitzer L2 Strom"
      voltage:
        name: "Durchlauferhitzer L2 Spannung"
      active_power:
        name: "Durchlauferhitzer L2 Wirkleistung"
      power_factor:
        name: "Durchlauferhitzer L2 Power Factor"
      apparent_power:
        name: "Durchlauferhitzer L2 Scheinleistung"
      reactive_power:
        name: "Durchlauferhitzer L2 Blindleistung"
      phase_angle:
        name: "Durchlauferhitzer L2 Phasenwinkel"
    phase_c:
      current:
        name: "Durchlauferhitzer L3 Strom"
      voltage:
        name: "Durchlauferhitzer L3 Spannung"
      active_power:
        name: "Durchlauferhitzer L3 Wirkleistung"
      power_factor:
        name: "Durchlauferhitzer L3 Power Factor"
      apparent_power:
        name: "Durchlauferhitzer L3 Scheinleistung"
      reactive_power:
        name: "Durchlauferhitzer L3 Blindleistung"
      phase_angle:
        name: "Durchlauferhitzer L3 Phasenwinkel"
    frequency:
      name: "Durchlauferhitzer Frequenz"
    total_power:
      name: "Durchlauferhitzer Gesamtleistung"
    import_active_energy:
      name: "Durchlauferhitzer Gesamtwirkarbeit"
    export_active_energy:
      name: "Durchlauferhitzer Gesamtwirkarbeit Export"
    import_reactive_energy:
      name: "Durchlauferhitzer Gesamtblindleistung"
    export_reactive_energy:
      name: "Durchlauferhitzer Gesamtblindleistung Export"
    update_interval: 3s

  - platform: sdm_meter
    address: 11
    phase_a:
      current:
        name: "Werkstatt L1 Strom"
      voltage:
        name: "Werkstatt L1 Spannung"
      active_power:
        name: "Werkstatt L1 Wirkleistung"
      power_factor:
        name: "Werkstatt L1 Power Factor"
      apparent_power:
        name: "Werkstatt L1 Scheinleistung"
      reactive_power:
        name: "Werkstatt L1 Blindleistung"
      phase_angle:
        name: "Werkstatt L1 Phasenwinkel"
    phase_b:
      current:
        name: "Werkstatt L2 Strom"
      voltage:
        name: "Werkstatt L2 Spannung"
      active_power:
        name: "Werkstatt L2 Wirkleistung"
      power_factor:
        name: "Werkstatt L2 Power Factor"
      apparent_power:
        name: "Werkstatt L2 Scheinleistung"
      reactive_power:
        name: "Werkstatt L2 Blindleistung"
      phase_angle:
        name: "Werkstatt L2 Phasenwinkel"
    phase_c:
      current:
        name: "Werkstatt L3 Strom"
      voltage:
        name: "Werkstatt L3 Spannung"
      active_power:
        name: "Werkstatt L3 Wirkleistung"
      power_factor:
        name: "Werkstatt L3 Power Factor"
      apparent_power:
        name: "Werkstatt L3 Scheinleistung"
      reactive_power:
        name: "Werkstatt L3 Blindleistung"
      phase_angle:
        name: "Werkstatt L3 Phasenwinkel"
    frequency:
      name: "Werkstatt Frequenz"
    total_power:
      name: "Werkstatt Gesamtleistung"
    import_active_energy:
      name: "Werkstatt Gesamtwirkarbeit"
    export_active_energy:
      name: "Werkstatt Gesamtwirkarbeit Export"
    import_reactive_energy:
      name: "Werkstatt Gesamtblindleistung"
    export_reactive_energy:
      name: "Werkstatt Gesamtblindleistung Export"
    update_interval: 29s

I would be grateful for any tips. All in all, it looks to me as if Grafana does not save any data that does not change. However, with other sensors (e.g. all current levels from the Shellys) this is saved and also displayed correctly.
Many thanks in advance.

It is not Grafana that is doing this, since Grafana is visualization only (generally speaking). This is HA and this IS precisely how it is defined to work (only save on change, presumably to save space on the disk). There might be a setting to change that (in HA, not in esphome). I recall seeing something about that, but can’t find it now.

I use MQTT, Node-RED, InfluxDB, and Grafana along with a mix of esphome and Tasmota. HA is great in giving you a semi-walled garden where things mostly work. It does allow you to get out when things don’t do what you want, but then you are typically in a wilderness area and you need strong survival skills.

Alright, thanks for your answer. Seems, like I am not the only one with those Problems: Improved handling of repeated sensor values (which are presently ignored) - #4 by boheme61
I’ll try to find a solution.

First, I’ll try the force_update attribute of the sensor. Maybe this will be enough. I didn’t know it before.

The argument is working fine. I’ll add it to a subset of recorded data sets. And I’ll have a look if Grafanas Query Arguments could do what I want without the need of recording every data point when it didn’t change at all.

I knew there was something, just didn’t remember where it was.

Grafana has an option (fill) that can use previous value for points that are missing. I usually use null or none (they are slightly different) so I can see where there is missing data.