Reduce number of DSMR 5 records to avoid huge database

I’m using the DSMR component to retrieve data (power consumption/production and gas consumption) from my ‘smart meter’. The smart meter uses the DSMR 5 protocol which updates every second. De home assistant db is growing fast and I want to reduce the number of entries written to the database. One record every 10 seconds or 1 minute is fine. Is that possible somehow?

1 Like

You should be able to use the filter sensor for this.

Of course, you also need to exclude the original sensor from the database

3 Likes

Great, thank you. The filter below will reduce the number of events each minute from 60 to 6 (every 10 seconds).

- platform: filter
  name: "<replace by your new sensor name>"
  entity_id: sensor.power_consumption
  filters:
  - filter: throttle
  - window_size: 10

I excluded sensor.power_consumption from the recorder and included the new sensor above. Now the database isn’t growing as fast anymore. :grinning:

1 Like

Another option is to solve the issue from the source by making a wireless readout device that connects to the MQTT broker of HA with an esp8266. I did it with a nodeMCU v1.0 which I had laying around.

Works perfectly and no hassle with the inverted TTL to USB communication by code and it’s WIRELESS. I recommend soldering the wires directly to the NodeMCU board.

Cheers.

1 Like

I’ve added some filters and excluded the same sensors from the recorder, restarted HA, but when I look at the webinterface of HA, I see values from the DSMR component still updating every second. I’m wondering if I’ve done things correctly :thinking:

Here’s my relevant configuration.yaml:

sensor: !include include/sensors.yaml
recorder: !include include/recorder.yaml

In my include/sensors.yaml I have:

  - platform: dsmr                                                                                                                                                                                          
    port: /dev/ttyUSB0                                                                                                                                                                                      
    dsmr_version: 5                                                                                                                                                                                         
                                                                                                                                                                                                            
  - platform: filter                                                                                                                                                                                        
    name: "Stroomverbruik laag tarief"                                                                                                                                                                      
    entity_id: sensor.power_consumption_low                                                                                                                                                                 
    filters:                                                                                                                                                                                                
      - filter: throttle                                                                                                                                                                                    
        window_size: 10                                                                                                                                                                                     
                                                                                                                                                                                                            
  - platform: filter                                                                                                                                                                                        
    name: "Stroomverbruik normaal tarief"                                                                                                                                                                   
    entity_id: sensor.power_consumption_normal                                                                                                                                                              
    filters:                                                                                                                                                                                                
      - filter: throttle                                                                                                                                                                                    
        window_size: 10                                                                                                                                                                                     
                                                                                                                                                                                                            
  - platform: filter                                                                                                                                                                                        
    name: "Gas verbruik"                                                                                                                                                                                    
    entity_id: sensor.gas_consumption                                                                                                                                                                       
    filters:                                                                                                                                                                                                
      - filter: throttle                                                                                                                                                                                    
        window_size: 10  

And in my include/recorder.yaml I have:

  exclude:                                                                                                                                                                                                  
    domains:                                                                                                                                                                                                
      - media_player                                                                                                                                                                                        
    entities:                                                                                                                                                                                               
      - sensor.power_consumption_low                                                                                                                                                                        
      - sensor.power_consumption_normal                                                                                                                                                                     
      - sensor.gas_consumption

I’ve restarted HA, but updates in number on the HA “Home” view are updating every minute. Have I missed something? :thinking:

– EDIT –
I believe, I’ve got my head crunching now and I see the picture. I’m creating new sensors with the filter, that update a lot less often. And in the meantime I’m disabling the sources of those new filters to be written to the HA recorder. I could now choose to hide those sources for the new filters from the “Home” view by entering them in customize.yaml with a hidden: true option.

Am I right?

And when I throttle and stop recording these sensors, are the newly created sensors also throttled to InfluxDB?

Hello Koen,

Does your setup needs external power? Like the USB.
Because i want to read out my meter, but my meter is in an central hall.

So i do not have access to an wall outlet. My meter is the: AM550-ED1

Greetings,
Jos

Hi Jos,

Unfortunately, the meters do not supply any suitable voltage to use that as a power source. You could look into:

  • Use a beefy power bank to power your esp for a while (would recommend desoldering the LEDs)
  • Esp8266 board with a rechargeable battery (maybe a 18615 battery) and spend some time into adding some extra features in the code to report the battery status, to keep track of that. WeMos D1 board

I think this is not the answer you were hoping for, but I hope this gives a possible solution for your issue.

Cheers,
Koen

Hello Koen,

Yes I was already afraid of that. I think I will use an powerbank and desolder the led.
One other question, do you know if this setup also works for the: ISKRA AM550?

Greetings,
Jos

Hey Jos,

I believe it should work, It is still the same protocol (DSMR5).
Sorry for the late reaction, I got no email of you response.

Cheers,
Koen

Hi Jos and Koen,

the DSMR has a 5V pin on the P1 port made specifically for powering a small/low-power reader.
i use a Wemos with ESP_Easy powerd from the P1 port, this connects to WiFi and opens a TCP port for network access to the serial port.
At the moment it has an uptime of 115 days, so there is no problem running a Wemos off the DSMR meter directly, without the need for a power source other then the smart meter itself.

good luck,
Dennis

1 Like

Hi Dennis,

Good to know, i am going to look into a P1 reader with tasmota soon, so this wil be a nice upgrade though.

Thnx,
Koen

Dennis,

Do you have a schema for the pins to the wemos board?

Thanks,

Reijer

I have the same setup. But I still see a every second reading of the sensors which I have excluded in the recorder.yaml. And this recording is also pushed to Influx.db database so I can see them in Grafana. The new sensors are working like they should. Every 10 seconds a reading and push to influx. I am worried about the recording of the every second sensors. I have all installed in HA on a SD card.What is going wrong?

In sensors.yaml I have:

#P1 meter
  - platform: dsmr
    port: /dev/ttyUSB0
    dsmr_version: 5
  - platform: filter
    name: Slimme meter stroom
    entity_id: sensor.power_consumption
    filters:
    - filter: throttle
      window_size: 10
  - platform: filter
    name: Slimme meter gas
    entity_id: sensor.gas_consumption
    filters:
    - filter: throttle
      window_size: 10

And in recorder.yaml I have:

exclude:
  entities:                                                                                                                                                                                               
    - sensor.power_consumption                                                                                                                                                                        
    - sensor.power_consumption_phase_l1 
    - sensor.power_consumption_phase_l2  
    - sensor.power_consumption_phase_l3                                                                                                                                                                      
    - sensor.power_production
    - sensor.power_production_phase_l1
    - sensor.power_production_phase_l2
    - sensor.power_production_phase_l3
    - sensor.long_power_failure_count
    - sensor.short_power_failure_count
    - sensor.gas_consumption
    - sensor.hourly_gas_consumption

Isn’t that what should happen though? You need the original sensors as the input for the new sensors.

As far as I can see, you are excluding them from the recorder. So, the original sensors should not be recorded to history. They’re only showing their current readings, which is what you need to actually record them into the new sensors (e.g. sensor.slimme_meter_gas), every 10 seconds instead of every second with the original sensors.

In 0.118, the DSMR integration will have an integrated throttle, to put a lower bound on the minimum time between entity updates :slight_smile:

1 Like

Cool, didn’t read that so thank you for pointing out that to me ( us ).
I see standard is 30 secs for new reading.

Hi guys,

Now the default is changed, I would like to ask how can I increase the readings.
The reason I’m asking this is that time to time I would like monitor the “current” in real time which is not really possible anymore. Every time after power up a device I need to wait up to 30 minutes to see the change in the related gauge card.
Thanks in advance.

Solved in

Hi
I just installed this slimme meter integration

  • Do i still have to exclude them in the recorder?
  • Do i need to create new sensors too?
    OR is this with latest version not an issue anymore?

So i put everything to another database outside HA? if so, how?
Anyone willing to share their dashboards showing their energy stuff