Support for reading Dutch Smart Meter (electricity/gas) (P1 port)

Whatever you do; post the outcome here please :slight_smile:

I think you can use this:

Looks promising. Had not thought of this. I will try this asap.

I have looked at the binary sensor and that is not what we need. The sensor is polled every minute. I actually need the REST notify or REST command. When I have the time I will try to modify the latter to support headers.

For the moment I have implemented the python script above. But I canā€™t get it to work, so I need some help. :cold_sweat:

I can successfully run the command from the command line in the virtualenv:

(homeassistant) homeassistant@raspberrypi:/home/pi$ python /home/homeassistant/.homeassistant/scripts/mindergas.py
20170329
{'reading': '2539.406', 'date': '20170329'}
<Response [201]>
Ready
(homeassistant) homeassistant@raspberrypi:/home/pi$ 

This is the configuration (I can see the logbook entry and the pushover notification, but the reading is not published, so the script is not being called, I guess.

sensor:
  - platform: dsmr
    #etc
shell_command:
  mindergas: 'python /home/homeassistant/.homeassistant/scripts/mindergas.py'
script:
  mindergas:
    alias: Verstuur gasverbruik
    sequence:
      - service: shell_command.mindergas
      - service: logbook.log
        data:
          name: Gasverbruik
          message: is verstuurd
          entity_id: sensor.gas_consumption
          domain: sensor
      - service: notify.pushover
        data_template:
          message: "Gasverbruik ({{ states.sensor.gas_consumption.state}} m3) is verstuurd naar mindergas.nl"
automation:
  mindergas:
    alias: Verstuur gasverbruik
    hide_entity: no # default no
    trigger:
      platform: time
      # run daily at 00:05:00
      hours: 00
      minutes: 15
      seconds: 00
    action:
      service: script.mindergas

I had the exact same problem! Thatā€™s why I decided to run the script through a cronjob instead of running it from Homeassistantā€¦

I think I have finally found the solution. To run a python script through a shell command in a virtual environment, you need to specify the full path to your python.

I have modified my shell_command to this (including redirecting output):

shell_command:
  mindergas: '/srv/homeassistant/bin/python /home/homeassistant/.homeassistant/scripts/mindergas.py >> /home/homeassistant/.homeassistant/mindergas.log'

And modified the script a little for pretty logging:

#!/usr/bin/env python
import requests
import json
import datetime
import homeassistant.remote as remote

mindergas = 'http://www.mindergas.nl/api/gas_meter_readings'
mindergas_token = 'XXXXXXXXXXXXXXXXXXXX'
hass_ip = '127.0.0.1'
hass_password = ''
hass_port = '8123'
gas_sensor = 'sensor.gas_consumption'

api = remote.API(hass_ip,hass_password, hass_port)
gas_usage = remote.get_state(api, 'sensor.gas_consumption')

def post():
    yesterday = datetime.date.today () - datetime.timedelta (days=1)
    data = {'date': yesterday.strftime ("%Y%m%d"), 'reading':gas_usage.state}
    headers = {'Content-Type': 'application/json', 'AUTH-TOKEN': mindergas_token}
    r = requests.post(mindergas, data=json.dumps(data), headers=headers)
    print (datetime.datetime.now(), '[INFO]', r.status_code, data)

post()

It seems to work now. I will monitor it the next few days.

Solution:

I have red the complete post but i have a few questions. First my setup:

  • Synology NAS 916+ 8GB with Home Assistant directly installed on it
  • Added module DSRM and it works.

and now my questions:

  • is it possible to decrease the time of gas? now every hour, i like 30 minutes for example
  • power production is only positive production? i have sunpanels and i see only now + and no -
  • is history available

thnx for your replies

Good to hear it works.

  • gas production interval is limited by when it gets send from the smartmeter which is unfortunately only every hour (might depend on meter)
  • what would be negative power production?
  • no, you need to record the history yourself using HA (recorder) by having HA push metrics to external db (graphite/influx/etc)

Some additions:

  • P1 ports send data every 10 or 1 seconds, depending on the DSMR version that it adheres to. P4 only sends gas each hour, true, but this is also done via the ā€˜P4ā€™ port, which is something entirely different from the P1 port.
  • Iā€™m assuming he meant he can see usage, but not what his PV installation generated: these are probably subtracted from each other by a system somewhere in the chain, the meter should give these two numbers separately in its P1 telegram.
  • This is true, the meter doesnā€™t record P1 data.

Thnx all for your supplies. I noticed already something

I only had the sensor power consumption and overlooked power production as a sensor. Now i see when my sunpanels are collecting and i they are producing too much, power production is the one i had to look at. When we are consuming, power consumptions number is shown. So that part i figured out.

Hourly Gas i already saw and i am fine with that. The question came because in domoticz is was almost real time.

Remains my question about history. What are you mening with HA recorder (Home Assistant Recorder?) to external db? And can i put those metrics in Home Assistant again?

Thanks

HA has a recorder module which stores metrics to a database which can later be viewed in HA again when history is also enabled (https://home-assistant.io/components/recorder/). There are also external metric databases that cannot be read by HA again but provide better integration with other tools, I would look at influxdb+grafana which is a really nice combination.

@DatBassie I donā€™t know about the P4 port, but I donā€™t think it is accessible for consumers. Otherwise if we can we should add support to the dsmr_parser project which in turn allows this component to use it.

Having solar power reduces the usefulness of the DSMR consumption metric. What is possible is if you also measure your solar output with HA to calculate the power usage using a template sensor, for example:

sensor:
  - platform: template
    sensors:
      netto_verbruik:
        friendly_name: Energie verbruik
        value_template: >
          {%  if states.sensor.zonenergie_verbruik.state and states.sensor.zonenergie_verbruik.state != "unknown" %}
          {{ states.sensor.power_consumption.state | float + states.sensor.zonenergie_verbruik.state | float }}
          {% else %}
          "unknown"
          {% endif %}
        unit_of_measurement: 'kW'
      zonenergie_verbruik:
        friendly_name: Zon energie verbruik
        value_template: >
          {%  if states.sensor.power_output.state and states.sensor.power_output.state != "unknown" %}
          {{ (states.sensor.power_output.state | float / 1000) - states.sensor.power_production.state | float }}
          {% else %}
          "unknown"
          {% endif %}
        unit_of_measurement: 'kW'

The snippet above produces two values, one for the amount of power from the solar panels that is used by devices in the house and one for the total power (solar + net) that is used by devices in the house (netto_verbruik). The last one would be the value of power used like if you had no solar panels, only net.

WoW thnx for your reply. I see your durch also :blush:.
So in this case it doesnā€™t matter what type of panels you have, youā€™re just reading the values.

I will try this script this evening or tomorrow. Or isnt it that easy to implement?

After that i will take a look af the history part

Thanks for the help

Hello aequitas,

question about your sensor example.
When i read your instruction i see you have ā€˜zonenergieā€™ sensors. But in my HA i dont have any sensors for my panels. So that said it isnt possible to use this example. Or am i wrong?

As i said, i have Solax-X inverter and it would be nice to implement those into HA, but i cant find any information about integration.

About history, it is already activated just like recorder. At this moment i monitor everything, i tried some include and exclude options but after restart i had no history at all. But youā€™re saying influxdb+grafana, thus with that you can export your HA db to this for nice graphs?

You need to have some HA component that does collect the solar output value indeed. If it does not have HA integration for your brand consider adding it yourself (nice way to learn programming) or try if it is possible with MQTT. Maybe there is a Solar-X program that can put values onto an MQTT bus which can then be read by HA?

The influx plugin will push all metrics HA receives into an influx database. Then you can read that database and make nice graphs using grafana. Other options are Graphite or OpenTSDB for the database. But influx is simplest I think.

You are correct, P4 is not available to the consumer.

The meter saves and sends this data to the grid operator. A third party (ODA) can then, if given permission, receive this data for further processing. This requires a fairly long and manual process on the userā€™s end of giving this ODA permission to use that userā€™s data.

Hello aequitas,

i have influxdb and grafana up and running on my synology through docker. Simple setup and now i have to figure out the data i want to use. So in that case i have a little study :smile:

For the solar output. I have not that much time to learn programming due to work, but i will search the internet for some solutions. If you read something, i hope you will share it here

I was quite disappointed when my resolution went from realtime to 30 minutes when they installed my smartmeter. I used to have a light sensor reading the last digit mirror for pulses.

Hello aequitas. I have a litte question about grafana. I can read the power consumption and production and have these as seperated graphs. But what i would like to see

  • both in 1 graph with colors
  • combine those two, i mean. For example. i used 10 kwh and deliverd 8, so my today is i used 2

Is this possible?

What port are you reading now and with what software? The P1 port on DSMR4 and lower sends data every 10 seconds, DSMR5+ specifies 1 second.

P4 electricity is every 15 minutes, gas is each hour.

I use P1 port on DSMR4
I am a little bit further already.

  • i have the two graphs in one graph now with automatically color difference
  • i have the same in a block graph
  • i have a table with power consumption and production

now i am searching for a way to combine the table with comsumption and production. So, at this moment i used 10, production is 2 so netto use is 8 kwh at the moment